Sep 05 06:25:51.519176 ip-10-0-128-174 systemd[1]: Starting Kubernetes Kubelet...
Sep 05 06:25:51.958012 ip-10-0-128-174 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 06:25:51.958012 ip-10-0-128-174 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 05 06:25:51.958012 ip-10-0-128-174 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 06:25:51.958012 ip-10-0-128-174 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 05 06:25:51.958012 ip-10-0-128-174 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 06:25:51.959004 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.958922    2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 05 06:25:51.964105 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964087    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:25:51.964105 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964103    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:25:51.964105 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964106    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:25:51.964105 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964109    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964113    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964116    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964119    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964122    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964127    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964131    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964134    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964137    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964140    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964147    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964150    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964153    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964156    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964160    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964163    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964166    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964169    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964172    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:25:51.964279 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964175    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964178    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964181    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964184    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964187    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964189    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964192    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964194    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964197    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964200    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964202    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964205    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964207    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964209    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964212    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964215    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964218    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964221    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964223    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964226    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:25:51.964951 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964228    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964231    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964234    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964237    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964239    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964242    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964245    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964247    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964250    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964252    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964255    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964257    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964260    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964262    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964265    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964267    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964270    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964272    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964276    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:25:51.965700 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964280    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964283    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964286    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964289    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964292    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964294    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964297    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964299    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964303    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964305    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964308    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964311    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964313    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964316    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964318    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964321    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964323    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964326    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964328    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964331    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:25:51.966388 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964333    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964336    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964338    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964341    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964343    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964345    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964350    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964352    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964355    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964357    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964359    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964362    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964364    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964367    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964370    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964372    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964374    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.964377    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965088    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965095    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:25:51.967103 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965099    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965102    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965105    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965109    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965112    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965115    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965118    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965120    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965123    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965126    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965129    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965132    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965134    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965137    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965140    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965142    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965145    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965147    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965150    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965153    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:25:51.967787 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965155    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965158    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965160    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965162    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965165    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965174    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965178    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965180    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965183    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965185    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965188    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965191    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965195    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965197    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965200    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965203    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965206    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965209    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965212    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965214    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:25:51.968515 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965217    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965221    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965223    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965226    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965228    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965231    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965233    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965237    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965241    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965245    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965249    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965252    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965255    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965258    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965261    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965263    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965266    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965269    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965272    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:25:51.969285 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965275    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965278    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965280    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965283    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965286    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965290    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965293    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965296    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965298    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965301    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965311    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965313    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965316    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965319    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965321    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965324    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965327    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965329    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965333    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965336    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:25:51.969985 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965338    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965341    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965344    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965346    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965350    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965353    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965356    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965359    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965362    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965365    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965367    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965370    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965373    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965376    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965379    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965382    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965384    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965387    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965758    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965763    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:25:51.970712 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965766    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965769    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965772    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965775    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965778    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965780    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965783    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965786    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965789    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965791    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965793    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965796    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965799    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965801    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965804    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965806    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965809    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965812    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965815    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965818    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:25:51.971477 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965820    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965823    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965825    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965828    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965830    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965833    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965836    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965838    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965841    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965845    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965861    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965864    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965866    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965869    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965871    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965874    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965877    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965880    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965882    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:25:51.972197 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965885    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965888    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965890    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965893    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965895    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965898    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965900    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965902    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965905    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965908    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965910    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965913    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965916    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965918    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965921    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965923    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965926    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965928    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965931    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965934    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:25:51.972902 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965939    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965941    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965944    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965948    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965951    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965954    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965958    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965960    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965964    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965967    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965970    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965972    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965975    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965978    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965980    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965983    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965985    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965988    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965990    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:25:51.973625 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965993    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965995    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.965999    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966002    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966004    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966007    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966010    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966012    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966015    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966019    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966021    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966024    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966027    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966031    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966033    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966036    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966040    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966043    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966046    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966127    2573 flags.go:64] FLAG: --address="0.0.0.0"
Sep 05 06:25:51.974301 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966161    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966167    2573 flags.go:64] FLAG: --anonymous-auth="true"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966171    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966176    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966180    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966184    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966189    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966192    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966195    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966198    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966202    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966205    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966208    2573 flags.go:64] FLAG: --cgroup-root=""
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966211    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966217    2573 flags.go:64] FLAG: --client-ca-file=""
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966219    2573 flags.go:64] FLAG: --cloud-provider="external"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966222    2573 flags.go:64] FLAG: --cluster-dns="[]"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966227    2573 flags.go:64] FLAG: --cluster-domain=""
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966230    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966233    2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966236    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966239    2573 flags.go:64] FLAG: --container-log-max-files="5"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966244    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 05 06:25:51.975032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966247    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966249    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966254    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966257    2573 flags.go:64] FLAG: --contention-profiling="false"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966260    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966263    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966266    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966269    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966273    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966277    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966279    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966282    2573 flags.go:64] FLAG: --enable-load-reader="false"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966285    2573 flags.go:64] FLAG: --enable-server="true"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966291    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966296    2573 flags.go:64] FLAG: --event-burst="100"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966299    2573 flags.go:64] FLAG: --event-qps="50"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966302    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966305    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966308    2573 flags.go:64] FLAG: --eviction-hard=""
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966312    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966315    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966318    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966321    2573 flags.go:64] FLAG: --eviction-soft=""
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966325    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966328    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 05 06:25:51.975829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966331    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966334    2573 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966337    2573 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966339    2573 flags.go:64] FLAG: --fail-swap-on="true"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966342    2573 flags.go:64] FLAG: --feature-gates=""
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966346    2573 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966349    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966352    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966355    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966360    2573 flags.go:64] FLAG: --healthz-port="10248"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966363    2573 flags.go:64] FLAG: --help="false"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966366    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-128-174.ec2.internal"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966369    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966372    2573 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966375    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966378    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966382    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966384    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966387    2573 flags.go:64] FLAG: --image-service-endpoint=""
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966390    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966393    2573 flags.go:64] FLAG: --kube-api-burst="100"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966396    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966400    2573 flags.go:64] FLAG: --kube-api-qps="50"
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966403    2573 flags.go:64] FLAG: --kube-reserved=""
Sep 05 06:25:51.976759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966405    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966408    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966412    2573 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966414    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966417    2573 flags.go:64] FLAG: --lock-file=""
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966420    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966423    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966427    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966432    2573 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966435    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966437    2573 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966440    2573 flags.go:64] FLAG: --logging-format="text"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966443    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966446    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966449    2573 flags.go:64] FLAG: --manifest-url=""
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966452    2573 flags.go:64] FLAG: --manifest-url-header=""
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966458    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966463    2573 flags.go:64] FLAG: --max-open-files="1000000"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966467    2573 flags.go:64] FLAG: --max-pods="110"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966470    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966473    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966476    2573 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966479    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966482    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966485    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 05 06:25:51.977771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966488    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966495    2573 flags.go:64] FLAG: --node-status-max-images="50"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966498    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966501    2573 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966504    2573 flags.go:64] FLAG: --pod-cidr=""
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966508    2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966513    2573 flags.go:64] FLAG: --pod-manifest-path=""
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966516    2573 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966519    2573 flags.go:64] FLAG: --pods-per-core="0"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966522    2573 flags.go:64] FLAG: --port="10250"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966525    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966528    2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06e8039d219697e85"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966531    2573 flags.go:64] FLAG: --qos-reserved=""
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966534    2573 flags.go:64] FLAG: --read-only-port="10255"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966537    2573 flags.go:64] FLAG: --register-node="true"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966540    2573 flags.go:64] FLAG: --register-with-taints=""
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966543    2573 flags.go:64] FLAG: --registry-burst="10"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966547    2573 flags.go:64] FLAG: --registry-qps="5"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966550    2573 flags.go:64] FLAG: --reserved-cpus=""
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966552    2573 flags.go:64] FLAG: --reserved-memory=""
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966556    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966559    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966562    2573 flags.go:64] FLAG: --rotate-certificates="false"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966565    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966569    2573 flags.go:64] FLAG: --runonce="false"
Sep 05 06:25:51.978695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966572    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966575    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966578    2573 flags.go:64] FLAG: --seccomp-default="false"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966581    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966584    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966586    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966590    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966593    2573 flags.go:64] FLAG: --storage-driver-password="root"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966596    2573 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966599    2573 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966601    2573 flags.go:64] FLAG: --storage-driver-user="root"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966604    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966607    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966610    2573 flags.go:64] FLAG: --system-cgroups=""
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966613    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966619    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966621    2573 flags.go:64] FLAG: --tls-cert-file=""
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966624    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966629    2573 flags.go:64] FLAG: --tls-min-version=""
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966631    2573 flags.go:64] FLAG: --tls-private-key-file=""
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966634    2573 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966637    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966640    2573 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966643    2573 flags.go:64] FLAG: --v="2"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966647    2573 flags.go:64] FLAG: --version="false"
Sep 05 06:25:51.979615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966651    2573 flags.go:64] FLAG: --vmodule=""
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966655    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.966658    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966773    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966778    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966781    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966786    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966788    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966791    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966794    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966796    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966799    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966801    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966804    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966807    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966809    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966812    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966815    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966817    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:25:51.980535 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966820    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966822    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966825    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966828    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966830    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966833    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966836    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966838    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966841    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966843    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966846    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966860    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966865    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966868    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966871    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966874    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966877    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966880    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966882    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966889    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:25:51.981246 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966892    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966895    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966897    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966900    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966903    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966905    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966908    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966910    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966913    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966915    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966918    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966921    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966923    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966926    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966929    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966932    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966934    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966937    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966939    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966942    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:25:51.981964 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966946    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966949    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966953    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966957    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966959    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966962    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966965    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966967    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966970    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966972    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966975    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966979    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966982    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966985    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966988    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966991    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966993    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966996    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.966998    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:25:51.982732 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967001    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967003    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967006    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967009    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967011    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967014    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967017    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967019    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967022    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967025    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967027    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967030    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967032    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967036    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967040    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967044    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967047    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967049    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967052    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967055    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:25:51.983553 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967057    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967060    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967062    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.967065    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.967072    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.975041    2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.975060    2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.975075    2573 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.975081    2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975145    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975150    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975154    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975158    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975160    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975163    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:25:51.984319 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975166    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975168    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975171    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975173    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975176    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975180    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975183    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975187    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975190    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975193    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975196    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975200    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975206    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975211    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975216    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975220    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975223    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975227    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:25:51.984913 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975231    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975234    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975237    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975241    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975244    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975248    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975250    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975253    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975256    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975259    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975261    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975264    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975267    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975270    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975272    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975275    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975278    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975281    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975286    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975290    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:25:51.985565 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975294    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975298    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975301    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975304    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975306    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975309    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975311    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975314    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975316    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975319    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975321    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975324    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975327    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975330    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975332    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975335    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975337    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975340    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975344    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975346    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:25:51.986310 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975349    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975351    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975354    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975356    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975360    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975364    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975368    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975372    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975377    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975379    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975382    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975384    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975387    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975389    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975392    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975394    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975397    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975399    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975402    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975405    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:25:51.987046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975407    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975410    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975413    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975416    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975419    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975421    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975424    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975427    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975430    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975432    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975434    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975439    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975443    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975447    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975451    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:25:51.987806 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.975457    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975579    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975585    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975588    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975591    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975595    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975599    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975603    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975607    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975611    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975614    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975617    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975619    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975622    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975624    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975626    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975629    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975631    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975634    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975636    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:25:51.988374 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975639    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975642    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975644    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975647    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975650    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975652    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975655    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975657    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975660    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975663    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975666    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975668    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975671    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975673    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975677    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975681    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975685    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975689    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975692    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975695    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:25:51.989178 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975697    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975700    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975702    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975705    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975707    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975710    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975712    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975715    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975717    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975720    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975722    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975725    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975728    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975732    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975735    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975738    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975741    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975744    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975747    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:25:51.989925 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975749    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975752    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975755    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975760    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975764    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975768    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975771    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975774    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975776    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975779    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975781    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975785    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975789    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975791    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975794    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975796    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975798    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975801    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975803    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975806    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:25:51.990614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975808    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975811    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975814    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975816    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975819    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975821    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975824    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975828    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975831    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975836    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975841    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975846    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975861    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975864    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975867    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975869    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975873    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975876    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975879    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:25:51.991349 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975882    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:25:51.992043 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:51.975884    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:25:51.992043 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.975889    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 06:25:51.992043 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.976021    2573 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 05 06:25:51.992043 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.979894    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 05 06:25:51.992043 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.980983    2573 server.go:1019] "Starting client certificate rotation"
Sep 05 06:25:51.992043 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.981082    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 05 06:25:51.992043 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.981158    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 05 06:25:51.992043 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.983798    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 05 06:25:51.992043 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.985035    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 05 06:25:51.998079 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:51.998046    2573 log.go:25] "Validated CRI v1 runtime API"
Sep 05 06:25:52.004025 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.004003    2573 log.go:25] "Validated CRI v1 image API"
Sep 05 06:25:52.004993 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.004975    2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 05 06:25:52.008546 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.008518    2573 fs.go:135] Filesystem UUIDs: map[4ae92075-cbca-47a2-80f9-85765c8dba0e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c1bface1-d7fc-4c27-9d14-896c7add52a4:/dev/nvme0n1p3]
Sep 05 06:25:52.008609 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.008544    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 05 06:25:52.015171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.015022    2573 manager.go:216] Machine: {Timestamp:2026-09-05 06:25:52.013432717 +0000 UTC m=+0.381479918 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102632 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d9765df03b01e1b0aa32b6ddcabcc SystemUUID:ec2d9765-df03-b01e-1b0a-a32b6ddcabcc BootID:bf9390c1-3657-43b6-9ef9-bb369e4d6908 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4a:00:b0:be:d7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4a:00:b0:be:d7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:7e:61:5e:2e:1a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 05 06:25:52.015171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.015160    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 05 06:25:52.015324 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.015310    2573 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 05 06:25:52.017591 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.017575    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 05 06:25:52.018084 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.018058    2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 05 06:25:52.018231 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.018090    2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-174.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 05 06:25:52.019045 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.019035    2573 topology_manager.go:138] "Creating topology manager with none policy"
Sep 05 06:25:52.019085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.019048    2573 container_manager_linux.go:309] "Creating device plugin manager"
Sep 05 06:25:52.019085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.019056    2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 05 06:25:52.019085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.019069    2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 05 06:25:52.021220 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.021207    2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 05 06:25:52.025922 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.025909    2573 state_mem.go:36] "Initialized new in-memory state store"
Sep 05 06:25:52.026043 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.026033    2573 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 05 06:25:52.026078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.026065    2573 kubelet.go:486] "Attempting to sync node with API server"
Sep 05 06:25:52.026110 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.026080    2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 05 06:25:52.026110 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.026104    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 05 06:25:52.026169 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.026120    2573 kubelet.go:398] "Adding apiserver pod source"
Sep 05 06:25:52.026169 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.026136    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 05 06:25:52.026554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.026541    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 05 06:25:52.026594 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.026567    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 05 06:25:52.030268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.030243    2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 05 06:25:52.033675 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.033656    2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 05 06:25:52.033761 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.033696    2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 05 06:25:52.033932 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.033920    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 05 06:25:52.033974 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.033939    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 05 06:25:52.033974 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.033949    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 05 06:25:52.033974 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.033957    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 05 06:25:52.033974 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.033968    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 05 06:25:52.033974 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.033975    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 05 06:25:52.034531 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.034521    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 05 06:25:52.034561 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.034533    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 05 06:25:52.034561 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.034542    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 05 06:25:52.034561 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.034550    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 05 06:25:52.034561 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.034560    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 05 06:25:52.034665 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.034572    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 05 06:25:52.035465 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.035449    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 05 06:25:52.035465 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.035463    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 05 06:25:52.039250 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.039234    2573 server.go:1268] "Started kubelet"
Sep 05 06:25:52.039361 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.039311    2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 05 06:25:52.039415 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.039384    2573 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 05 06:25:52.039474 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.039331    2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 05 06:25:52.039931 ip-10-0-128-174 systemd[1]: Started Kubernetes Kubelet.
Sep 05 06:25:52.041046 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.041018    2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 05 06:25:52.042066 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.042048    2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-174.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 05 06:25:52.042276 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.042255    2573 server.go:310] "Adding debug handlers to kubelet server"
Sep 05 06:25:52.042781 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.042751    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 05 06:25:52.042781 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.042754    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-174.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 05 06:25:52.045116 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.045082    2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 05 06:25:52.046763 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.046709    2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 05 06:25:52.046763 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.046727    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 05 06:25:52.046970 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.046952    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:52.047537 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.047523    2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 05 06:25:52.047631 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.047541    2573 factory.go:55] Registering systemd factory
Sep 05 06:25:52.047631 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.047574    2573 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 05 06:25:52.047631 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.047587    2573 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 05 06:25:52.047631 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.047599    2573 factory.go:223] Registration of the systemd container factory successfully
Sep 05 06:25:52.047807 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.047769    2573 reconstruct.go:97] "Volume reconstruction finished"
Sep 05 06:25:52.047807 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.047778    2573 reconciler.go:29] "Reconciler: start to sync state"
Sep 05 06:25:52.047971 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.047955    2573 factory.go:153] Registering CRI-O factory
Sep 05 06:25:52.048026 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.047973    2573 factory.go:223] Registration of the crio container factory successfully
Sep 05 06:25:52.048078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.048049    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 05 06:25:52.048078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.048068    2573 factory.go:103] Registering Raw factory
Sep 05 06:25:52.048181 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.048082    2573 manager.go:1196] Started watching for new ooms in manager
Sep 05 06:25:52.049182 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.049157    2573 manager.go:349] Starting recovery of all containers
Sep 05 06:25:52.049343 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.048010    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-174.ec2.internal.18d258b8a2459dc0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-174.ec2.internal,UID:ip-10-0-128-174.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-174.ec2.internal,},FirstTimestamp:2026-09-05 06:25:52.039206336 +0000 UTC m=+0.407253536,LastTimestamp:2026-09-05 06:25:52.039206336 +0000 UTC m=+0.407253536,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-174.ec2.internal,}"
Sep 05 06:25:52.053208 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.053180    2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-174.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 05 06:25:52.053291 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.053204    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 05 06:25:52.062421 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.062363    2573 manager.go:354] Recovery completed
Sep 05 06:25:52.067278 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.067259    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:25:52.069079 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.069060    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8wgcf"
Sep 05 06:25:52.071578 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.071560    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:25:52.071659 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.071594    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:25:52.071659 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.071609    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:25:52.072105 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.072091    2573 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 05 06:25:52.072105 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.072104    2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 05 06:25:52.072188 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.072120    2573 state_mem.go:36] "Initialized new in-memory state store"
Sep 05 06:25:52.072188 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.072139    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:25:52.072909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.072893    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:25:52.072980 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.072920    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:25:52.072980 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.072932    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:25:52.074030 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.073960    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-174.ec2.internal.18d258b8a433944a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-174.ec2.internal,UID:ip-10-0-128-174.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-174.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-174.ec2.internal,},FirstTimestamp:2026-09-05 06:25:52.071578698 +0000 UTC m=+0.439625899,LastTimestamp:2026-09-05 06:25:52.071578698 +0000 UTC m=+0.439625899,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-174.ec2.internal,}"
Sep 05 06:25:52.074327 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.074315    2573 policy_none.go:49] "None policy: Start"
Sep 05 06:25:52.074383 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.074330    2573 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 05 06:25:52.074383 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.074340    2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 05 06:25:52.076352 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.076338    2573 policy_none.go:47] "Start"
Sep 05 06:25:52.078776 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.078759    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8wgcf"
Sep 05 06:25:52.112573 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.112548    2573 manager.go:337] "Starting Device Plugin manager"
Sep 05 06:25:52.127555 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.112615    2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 05 06:25:52.127555 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.112629    2573 server.go:87] "Starting device plugin registration server"
Sep 05 06:25:52.127555 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.112899    2573 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 05 06:25:52.127555 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.112910    2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 05 06:25:52.127555 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.113014    2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 05 06:25:52.127555 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.113087    2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 05 06:25:52.127555 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.113096    2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 05 06:25:52.127555 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.113598    2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 05 06:25:52.127555 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.113660    2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:52.147805 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.147765    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 05 06:25:52.149048 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.149035    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 05 06:25:52.149117 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.149085    2573 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 05 06:25:52.149117 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.149109    2573 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 05 06:25:52.149224 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.149145    2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 05 06:25:52.151721 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.151702    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 06:25:52.213867 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.213720    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:25:52.214894 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.214874    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:25:52.215034 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.214911    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:25:52.215034 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.214924    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:25:52.215034 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.214954    2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.224544 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.224518    2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.224659 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.224553    2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-174.ec2.internal\": node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:52.249820 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.249779    2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-174.ec2.internal"]
Sep 05 06:25:52.250001 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.249870    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:25:52.250841 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.250824    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:25:52.250907 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.250878    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:25:52.250907 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.250896    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:25:52.253310 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.253297    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:25:52.253465 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.253438    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.253531 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.253486    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:25:52.254117 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.254100    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:25:52.254232 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.254133    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:25:52.254232 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.254158    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:25:52.254545 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.254527    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:25:52.254646 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.254565    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:25:52.254646 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.254576    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:25:52.256676 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.256657    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:52.257148 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.257133    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.257236 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.257157    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:25:52.257983 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.257968    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:25:52.258067 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.258004    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:25:52.258067 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.258019    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:25:52.279642 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.279607    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-174.ec2.internal\" not found" node="ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.283953 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.283933    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-174.ec2.internal\" not found" node="ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.348896 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.348864    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d469df84f974ddd8b8974ad2aded20b7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal\" (UID: \"d469df84f974ddd8b8974ad2aded20b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.349015 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.348895    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d469df84f974ddd8b8974ad2aded20b7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal\" (UID: \"d469df84f974ddd8b8974ad2aded20b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.349015 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.348938    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ddc0eaee28eb4599cb47316bccad4632-config\") pod \"kube-apiserver-proxy-ip-10-0-128-174.ec2.internal\" (UID: \"ddc0eaee28eb4599cb47316bccad4632\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.356971 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.356954    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:52.449510 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.449481    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d469df84f974ddd8b8974ad2aded20b7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal\" (UID: \"d469df84f974ddd8b8974ad2aded20b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.449614 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.449520    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ddc0eaee28eb4599cb47316bccad4632-config\") pod \"kube-apiserver-proxy-ip-10-0-128-174.ec2.internal\" (UID: \"ddc0eaee28eb4599cb47316bccad4632\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.449614 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.449542    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d469df84f974ddd8b8974ad2aded20b7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal\" (UID: \"d469df84f974ddd8b8974ad2aded20b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.449614 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.449545    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d469df84f974ddd8b8974ad2aded20b7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal\" (UID: \"d469df84f974ddd8b8974ad2aded20b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.449614 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.449582    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ddc0eaee28eb4599cb47316bccad4632-config\") pod \"kube-apiserver-proxy-ip-10-0-128-174.ec2.internal\" (UID: \"ddc0eaee28eb4599cb47316bccad4632\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.449614 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.449595    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d469df84f974ddd8b8974ad2aded20b7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal\" (UID: \"d469df84f974ddd8b8974ad2aded20b7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.457146 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.457110    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:52.557683 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.557599    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:52.582617 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.582573    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.586342 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.586315    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:52.659014 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.658787    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:52.759385 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.759342    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:52.860464 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.860381    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:52.960786 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:52.960744    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:52.981102 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.981072    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 05 06:25:52.981268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:52.981249    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 05 06:25:53.061432 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:53.061399    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:53.081996 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:53.081939    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-04 06:20:52 +0000 UTC" deadline="2028-02-13 07:03:58.413819899 +0000 UTC"
Sep 05 06:25:53.081996 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:53.081991    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12624h38m5.331833185s"
Sep 05 06:25:53.149080 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:53.148996    2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 06:25:53.161676 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:53.161640    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:53.207439 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:53.207388    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd469df84f974ddd8b8974ad2aded20b7.slice/crio-efee09d1f5f7261967d2163e1768e8d1acd1737cf0471cbd6301ddb97c9b5afe WatchSource:0}: Error finding container efee09d1f5f7261967d2163e1768e8d1acd1737cf0471cbd6301ddb97c9b5afe: Status 404 returned error can't find the container with id efee09d1f5f7261967d2163e1768e8d1acd1737cf0471cbd6301ddb97c9b5afe
Sep 05 06:25:53.207637 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:53.207620    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc0eaee28eb4599cb47316bccad4632.slice/crio-b80ecd6ed7c86fd1226c64673aae62e6481c3bc82174048dcd422b27c63f2434 WatchSource:0}: Error finding container b80ecd6ed7c86fd1226c64673aae62e6481c3bc82174048dcd422b27c63f2434: Status 404 returned error can't find the container with id b80ecd6ed7c86fd1226c64673aae62e6481c3bc82174048dcd422b27c63f2434
Sep 05 06:25:53.212638 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:53.212615    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 06:25:53.262169 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:53.262113    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:53.321657 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:53.321628    2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 06:25:53.362433 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:53.362387    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:53.463303 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:53.463202    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-174.ec2.internal\" not found"
Sep 05 06:25:53.540430 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:53.540394    2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 06:25:53.547336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:53.547306    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:53.572044 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:53.572010    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 05 06:25:53.572982 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:53.572967    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-174.ec2.internal"
Sep 05 06:25:53.581945 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:53.581920    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 05 06:25:54.027764 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.027721    2573 apiserver.go:52] "Watching apiserver"
Sep 05 06:25:54.040954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.040751    2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 05 06:25:54.041941 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.041906    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-174.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz","openshift-dns/node-resolver-nq8t2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal","openshift-multus/multus-tznq4","openshift-ovn-kubernetes/ovnkube-node-gbmg6","openshift-cluster-node-tuning-operator/tuned-ggtx7","openshift-image-registry/node-ca-7pdxl","openshift-multus/multus-additional-cni-plugins-tdcbd","openshift-multus/network-metrics-daemon-75v4r","openshift-network-diagnostics/network-check-target-76rz9","openshift-network-operator/iptables-alerter-mx24p","kube-system/global-pull-secret-syncer-cjd64","kube-system/konnectivity-agent-8xv7j"]
Sep 05 06:25:54.045088 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.044690    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:25:54.045088 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.044791    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:25:54.047221 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.047194    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:25:54.047349 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.047276    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:25:54.048690 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.048319    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 05 06:25:54.049643 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.049623    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nq8t2"
Sep 05 06:25:54.053075 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.051968    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7pdxl"
Sep 05 06:25:54.053441 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.053420    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lv4l9\""
Sep 05 06:25:54.054445 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.054423    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.057860 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.057839    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 05 06:25:54.057972 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.057884    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7pqgb\""
Sep 05 06:25:54.057972 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.057936    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 05 06:25:54.058119 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.058102    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 05 06:25:54.058189 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.058114    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 05 06:25:54.058189 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.058144    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mx24p"
Sep 05 06:25:54.058189 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.058176    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 05 06:25:54.058451 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.058431    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 05 06:25:54.058577 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.058479    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 05 06:25:54.058577 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.058496    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-h7dgb\""
Sep 05 06:25:54.058577 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.058433    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 05 06:25:54.060767 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.060743    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7095477-5141-481a-8b27-64e4c36845cb-tmp-dir\") pod \"node-resolver-nq8t2\" (UID: \"e7095477-5141-481a-8b27-64e4c36845cb\") " pod="openshift-dns/node-resolver-nq8t2"
Sep 05 06:25:54.060895 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.060774    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:25:54.060895 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.060792    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmnkk\" (UniqueName: \"kubernetes.io/projected/e75f6f78-89ec-47e6-a11c-910a4362182d-kube-api-access-jmnkk\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:25:54.060895 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.060833    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t47pn\" (UniqueName: \"kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn\") pod \"network-check-target-76rz9\" (UID: \"24c5e798-bf33-4f1d-8296-6c4bcff878d6\") " pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:25:54.061257 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.060908    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e7095477-5141-481a-8b27-64e4c36845cb-hosts-file\") pod \"node-resolver-nq8t2\" (UID: \"e7095477-5141-481a-8b27-64e4c36845cb\") " pod="openshift-dns/node-resolver-nq8t2"
Sep 05 06:25:54.061257 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.060957    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2l46\" (UniqueName: \"kubernetes.io/projected/e7095477-5141-481a-8b27-64e4c36845cb-kube-api-access-k2l46\") pod \"node-resolver-nq8t2\" (UID: \"e7095477-5141-481a-8b27-64e4c36845cb\") " pod="openshift-dns/node-resolver-nq8t2"
Sep 05 06:25:54.061257 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.061102    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.061853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.061830    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kz7bm\""
Sep 05 06:25:54.062511 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.062375    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 05 06:25:54.062727 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.062708    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 05 06:25:54.062988 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.062973    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 05 06:25:54.063262 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.063208    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 05 06:25:54.063485 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.063465    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.065609 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.065590    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.066265 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.066238    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 05 06:25:54.066382 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.066365    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 05 06:25:54.066491 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.066475    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gsqzl\""
Sep 05 06:25:54.068294 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.068272    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:54.068392 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.068345    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:25:54.070172 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.070152    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 05 06:25:54.071101 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.071080    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.071516 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.071497    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 05 06:25:54.071712 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.071696    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 05 06:25:54.072278 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.072252    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-l9cqq\""
Sep 05 06:25:54.072370 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.072293    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 05 06:25:54.072512 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.072495    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 05 06:25:54.072584 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.072514    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-njcnk\""
Sep 05 06:25:54.072740 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.072721    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 05 06:25:54.073687 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.073665    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:25:54.076726 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.076704    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 05 06:25:54.077762 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.077742    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 05 06:25:54.077913 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.077879    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 05 06:25:54.080660 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.079172    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 05 06:25:54.080660 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.079627    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2gjp5\""
Sep 05 06:25:54.080660 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.079850    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 05 06:25:54.080660 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.080043    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 05 06:25:54.080660 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.080098    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8x8kl\""
Sep 05 06:25:54.080660 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.080270    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 05 06:25:54.080660 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.080509    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 05 06:25:54.089354 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.089331    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-m9vwj"
Sep 05 06:25:54.099462 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.099416    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-m9vwj"
Sep 05 06:25:54.149372 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.149333    2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 05 06:25:54.155176 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.155114    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-174.ec2.internal" event={"ID":"ddc0eaee28eb4599cb47316bccad4632","Type":"ContainerStarted","Data":"b80ecd6ed7c86fd1226c64673aae62e6481c3bc82174048dcd422b27c63f2434"}
Sep 05 06:25:54.156744 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.156712    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal" event={"ID":"d469df84f974ddd8b8974ad2aded20b7","Type":"ContainerStarted","Data":"efee09d1f5f7261967d2163e1768e8d1acd1737cf0471cbd6301ddb97c9b5afe"}
Sep 05 06:25:54.161746 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161706    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-tmp\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.161879 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161752    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02c25fef-4296-4319-ba54-3ad5d77df1ca-env-overrides\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.161879 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161779    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx9lv\" (UniqueName: \"kubernetes.io/projected/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-kube-api-access-vx9lv\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.161879 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161802    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2471243c-2dff-4f8f-a174-8ddf1264dacf-iptables-alerter-script\") pod \"iptables-alerter-mx24p\" (UID: \"2471243c-2dff-4f8f-a174-8ddf1264dacf\") " pod="openshift-network-operator/iptables-alerter-mx24p"
Sep 05 06:25:54.161879 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161819    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2471243c-2dff-4f8f-a174-8ddf1264dacf-host-slash\") pod \"iptables-alerter-mx24p\" (UID: \"2471243c-2dff-4f8f-a174-8ddf1264dacf\") " pod="openshift-network-operator/iptables-alerter-mx24p"
Sep 05 06:25:54.161879 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161841    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-cni-netd\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.162122 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161864    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgppn\" (UniqueName: \"kubernetes.io/projected/02c25fef-4296-4319-ba54-3ad5d77df1ca-kube-api-access-bgppn\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.162122 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161908    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8cae7568-294a-4c0e-adf9-d9581668c54e-konnectivity-ca\") pod \"konnectivity-agent-8xv7j\" (UID: \"8cae7568-294a-4c0e-adf9-d9581668c54e\") " pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:25:54.162122 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161924    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-lib-modules\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.162122 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161947    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-log-socket\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.162122 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161971    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8cae7568-294a-4c0e-adf9-d9581668c54e-agent-certs\") pod \"konnectivity-agent-8xv7j\" (UID: \"8cae7568-294a-4c0e-adf9-d9581668c54e\") " pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:25:54.162122 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.161995    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bad5cfd-a364-42c1-90dd-e2f7ddc39e64-host\") pod \"node-ca-7pdxl\" (UID: \"4bad5cfd-a364-42c1-90dd-e2f7ddc39e64\") " pod="openshift-image-registry/node-ca-7pdxl"
Sep 05 06:25:54.162122 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162066    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-hostroot\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.162122 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162108    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-etc-selinux\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162138    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-run-ovn-kubernetes\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162164    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-socket-dir-parent\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162190    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-run-k8s-cni-cncf-io\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162218    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lncbm\" (UniqueName: \"kubernetes.io/projected/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-kube-api-access-lncbm\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162241    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac5d72db-04fc-4fbc-9db7-959c7458ad90-dbus\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162268    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162300    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-etc-openvswitch\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162327    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-run-ovn\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162355    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162384    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-sys-fs\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162409    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2jpp\" (UniqueName: \"kubernetes.io/projected/2471243c-2dff-4f8f-a174-8ddf1264dacf-kube-api-access-b2jpp\") pod \"iptables-alerter-mx24p\" (UID: \"2471243c-2dff-4f8f-a174-8ddf1264dacf\") " pod="openshift-network-operator/iptables-alerter-mx24p"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162436    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-modprobe-d\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162462    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-cni-binary-copy\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162488    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-kubelet\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.162526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162527    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162551    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-tuned\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162577    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-node-log\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162602    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/774d1813-9ee9-43c7-b398-bdb63b7f7e86-system-cni-dir\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162634    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/774d1813-9ee9-43c7-b398-bdb63b7f7e86-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162676    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/774d1813-9ee9-43c7-b398-bdb63b7f7e86-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162702    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-host\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162727    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-run-netns\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162751    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-run-openvswitch\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162777    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-var-lib-cni-multus\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162807    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-device-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162833    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-systemd-units\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162867    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-run-systemd\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162938    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02c25fef-4296-4319-ba54-3ad5d77df1ca-ovn-node-metrics-cert\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162971    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-etc-kubernetes\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.162997    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-cni-bin\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.163287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163024    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-system-cni-dir\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163048    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-run-multus-certs\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163076    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac5d72db-04fc-4fbc-9db7-959c7458ad90-kubelet-config\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163107    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-kubernetes\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163142    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-sysctl-conf\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163166    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-sys\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163192    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bad5cfd-a364-42c1-90dd-e2f7ddc39e64-serviceca\") pod \"node-ca-7pdxl\" (UID: \"4bad5cfd-a364-42c1-90dd-e2f7ddc39e64\") " pod="openshift-image-registry/node-ca-7pdxl"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163216    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-run-netns\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163241    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-daemon-config\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163268    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/774d1813-9ee9-43c7-b398-bdb63b7f7e86-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163297    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-slash\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163324    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02c25fef-4296-4319-ba54-3ad5d77df1ca-ovnkube-config\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163351    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-var-lib-kubelet\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163377    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-conf-dir\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163403    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/774d1813-9ee9-43c7-b398-bdb63b7f7e86-cni-binary-copy\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163433    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgxzq\" (UniqueName: \"kubernetes.io/projected/774d1813-9ee9-43c7-b398-bdb63b7f7e86-kube-api-access-lgxzq\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.164096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163464    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmnkk\" (UniqueName: \"kubernetes.io/projected/e75f6f78-89ec-47e6-a11c-910a4362182d-kube-api-access-jmnkk\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163495    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t47pn\" (UniqueName: \"kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn\") pod \"network-check-target-76rz9\" (UID: \"24c5e798-bf33-4f1d-8296-6c4bcff878d6\") " pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163522    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-sysctl-d\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163550    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-var-lib-cni-bin\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163576    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-socket-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163609    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7095477-5141-481a-8b27-64e4c36845cb-tmp-dir\") pod \"node-resolver-nq8t2\" (UID: \"e7095477-5141-481a-8b27-64e4c36845cb\") " pod="openshift-dns/node-resolver-nq8t2"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163657    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-systemd\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163683    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02c25fef-4296-4319-ba54-3ad5d77df1ca-ovnkube-script-lib\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163708    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-cni-dir\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163735    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-cnibin\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163762    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/774d1813-9ee9-43c7-b398-bdb63b7f7e86-cnibin\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163788    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163815    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-registration-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163842    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qscsv\" (UniqueName: \"kubernetes.io/projected/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-kube-api-access-qscsv\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163870    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e7095477-5141-481a-8b27-64e4c36845cb-hosts-file\") pod \"node-resolver-nq8t2\" (UID: \"e7095477-5141-481a-8b27-64e4c36845cb\") " pod="openshift-dns/node-resolver-nq8t2"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163902    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2l46\" (UniqueName: \"kubernetes.io/projected/e7095477-5141-481a-8b27-64e4c36845cb-kube-api-access-k2l46\") pod \"node-resolver-nq8t2\" (UID: \"e7095477-5141-481a-8b27-64e4c36845cb\") " pod="openshift-dns/node-resolver-nq8t2"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163946    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmdqn\" (UniqueName: \"kubernetes.io/projected/4bad5cfd-a364-42c1-90dd-e2f7ddc39e64-kube-api-access-cmdqn\") pod \"node-ca-7pdxl\" (UID: \"4bad5cfd-a364-42c1-90dd-e2f7ddc39e64\") " pod="openshift-image-registry/node-ca-7pdxl"
Sep 05 06:25:54.164880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.163974    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-sysconfig\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.165733 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.164000    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-run\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.165733 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.164028    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-var-lib-kubelet\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.165733 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.164056    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-var-lib-openvswitch\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.165733 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.164241    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:25:54.165733 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.164329    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs podName:e75f6f78-89ec-47e6-a11c-910a4362182d nodeName:}" failed. No retries permitted until 2026-09-05 06:25:54.664291793 +0000 UTC m=+3.032339001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs") pod "network-metrics-daemon-75v4r" (UID: "e75f6f78-89ec-47e6-a11c-910a4362182d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:25:54.165733 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.165570    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7095477-5141-481a-8b27-64e4c36845cb-tmp-dir\") pod \"node-resolver-nq8t2\" (UID: \"e7095477-5141-481a-8b27-64e4c36845cb\") " pod="openshift-dns/node-resolver-nq8t2"
Sep 05 06:25:54.166073 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.165750    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e7095477-5141-481a-8b27-64e4c36845cb-hosts-file\") pod \"node-resolver-nq8t2\" (UID: \"e7095477-5141-481a-8b27-64e4c36845cb\") " pod="openshift-dns/node-resolver-nq8t2"
Sep 05 06:25:54.173730 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.173704    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:25:54.173730 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.173728    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:25:54.173948 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.173744    2573 projected.go:196] Error preparing data for projected volume kube-api-access-t47pn for pod openshift-network-diagnostics/network-check-target-76rz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:25:54.173948 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.173817    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn podName:24c5e798-bf33-4f1d-8296-6c4bcff878d6 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:54.67379679 +0000 UTC m=+3.041843998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t47pn" (UniqueName: "kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn") pod "network-check-target-76rz9" (UID: "24c5e798-bf33-4f1d-8296-6c4bcff878d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:25:54.174089 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.174070    2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 05 06:25:54.181279 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.178732    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2l46\" (UniqueName: \"kubernetes.io/projected/e7095477-5141-481a-8b27-64e4c36845cb-kube-api-access-k2l46\") pod \"node-resolver-nq8t2\" (UID: \"e7095477-5141-481a-8b27-64e4c36845cb\") " pod="openshift-dns/node-resolver-nq8t2"
Sep 05 06:25:54.181279 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.179136    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmnkk\" (UniqueName: \"kubernetes.io/projected/e75f6f78-89ec-47e6-a11c-910a4362182d-kube-api-access-jmnkk\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:25:54.264600 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264562    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-cni-bin\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.264600 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264602    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-system-cni-dir\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264625    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-run-multus-certs\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264682    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac5d72db-04fc-4fbc-9db7-959c7458ad90-kubelet-config\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264709    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-kubernetes\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264709    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-cni-bin\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264724    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-system-cni-dir\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264713    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-run-multus-certs\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264771    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac5d72db-04fc-4fbc-9db7-959c7458ad90-kubelet-config\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264787    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-kubernetes\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264734    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-sysctl-conf\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264839    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-sys\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264866    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-sysctl-conf\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.264853 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264882    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bad5cfd-a364-42c1-90dd-e2f7ddc39e64-serviceca\") pod \"node-ca-7pdxl\" (UID: \"4bad5cfd-a364-42c1-90dd-e2f7ddc39e64\") " pod="openshift-image-registry/node-ca-7pdxl"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264915    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-run-netns\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264920    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-sys\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264937    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-daemon-config\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264969    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-run-netns\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.264974    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/774d1813-9ee9-43c7-b398-bdb63b7f7e86-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265002    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-slash\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265027    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02c25fef-4296-4319-ba54-3ad5d77df1ca-ovnkube-config\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265055    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-var-lib-kubelet\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265079    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-conf-dir\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265093    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-slash\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265104    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/774d1813-9ee9-43c7-b398-bdb63b7f7e86-cni-binary-copy\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265133    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgxzq\" (UniqueName: \"kubernetes.io/projected/774d1813-9ee9-43c7-b398-bdb63b7f7e86-kube-api-access-lgxzq\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265192    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-sysctl-d\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265218    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-var-lib-cni-bin\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265242    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-socket-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265265    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-systemd\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.265519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265288    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02c25fef-4296-4319-ba54-3ad5d77df1ca-ovnkube-script-lib\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265312    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-cni-dir\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265336    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-cnibin\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265363    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/774d1813-9ee9-43c7-b398-bdb63b7f7e86-cnibin\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265373    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bad5cfd-a364-42c1-90dd-e2f7ddc39e64-serviceca\") pod \"node-ca-7pdxl\" (UID: \"4bad5cfd-a364-42c1-90dd-e2f7ddc39e64\") " pod="openshift-image-registry/node-ca-7pdxl"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265393    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265425    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-registration-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265508    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/774d1813-9ee9-43c7-b398-bdb63b7f7e86-cnibin\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265534    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qscsv\" (UniqueName: \"kubernetes.io/projected/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-kube-api-access-qscsv\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265553    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmdqn\" (UniqueName: \"kubernetes.io/projected/4bad5cfd-a364-42c1-90dd-e2f7ddc39e64-kube-api-access-cmdqn\") pod \"node-ca-7pdxl\" (UID: \"4bad5cfd-a364-42c1-90dd-e2f7ddc39e64\") " pod="openshift-image-registry/node-ca-7pdxl"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265570    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-sysconfig\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265574    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-daemon-config\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265587    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-run\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265604    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-var-lib-kubelet\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265610    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02c25fef-4296-4319-ba54-3ad5d77df1ca-ovnkube-config\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265620    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-var-lib-openvswitch\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265638    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-tmp\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265639    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-systemd\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.266199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265654    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02c25fef-4296-4319-ba54-3ad5d77df1ca-env-overrides\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.265823    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-registration-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266206    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-sysctl-d\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266224    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-socket-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266396    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266550    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-cni-dir\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266599    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-var-lib-openvswitch\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266631    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-sysconfig\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266645    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02c25fef-4296-4319-ba54-3ad5d77df1ca-env-overrides\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266681    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-var-lib-kubelet\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266764    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-var-lib-cni-bin\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266663    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-run\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266826    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-cnibin\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.266843    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx9lv\" (UniqueName: \"kubernetes.io/projected/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-kube-api-access-vx9lv\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.267053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267052    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-conf-dir\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.267755 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267096    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2471243c-2dff-4f8f-a174-8ddf1264dacf-iptables-alerter-script\") pod \"iptables-alerter-mx24p\" (UID: \"2471243c-2dff-4f8f-a174-8ddf1264dacf\") " pod="openshift-network-operator/iptables-alerter-mx24p"
Sep 05 06:25:54.267755 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267167    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-var-lib-kubelet\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.267755 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267233    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2471243c-2dff-4f8f-a174-8ddf1264dacf-host-slash\") pod \"iptables-alerter-mx24p\" (UID: \"2471243c-2dff-4f8f-a174-8ddf1264dacf\") " pod="openshift-network-operator/iptables-alerter-mx24p"
Sep 05 06:25:54.267755 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267266    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-cni-netd\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.267755 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267529    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02c25fef-4296-4319-ba54-3ad5d77df1ca-ovnkube-script-lib\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.267755 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267546    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgppn\" (UniqueName: \"kubernetes.io/projected/02c25fef-4296-4319-ba54-3ad5d77df1ca-kube-api-access-bgppn\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.267755 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267584    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8cae7568-294a-4c0e-adf9-d9581668c54e-konnectivity-ca\") pod \"konnectivity-agent-8xv7j\" (UID: \"8cae7568-294a-4c0e-adf9-d9581668c54e\") " pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:25:54.267755 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267593    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-cni-netd\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.267755 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267617    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-lib-modules\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.268242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267777    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2471243c-2dff-4f8f-a174-8ddf1264dacf-iptables-alerter-script\") pod \"iptables-alerter-mx24p\" (UID: \"2471243c-2dff-4f8f-a174-8ddf1264dacf\") " pod="openshift-network-operator/iptables-alerter-mx24p"
Sep 05 06:25:54.268242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267808    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-log-socket\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.268242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267853    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2471243c-2dff-4f8f-a174-8ddf1264dacf-host-slash\") pod \"iptables-alerter-mx24p\" (UID: \"2471243c-2dff-4f8f-a174-8ddf1264dacf\") " pod="openshift-network-operator/iptables-alerter-mx24p"
Sep 05 06:25:54.268242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267859    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8cae7568-294a-4c0e-adf9-d9581668c54e-agent-certs\") pod \"konnectivity-agent-8xv7j\" (UID: \"8cae7568-294a-4c0e-adf9-d9581668c54e\") " pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:25:54.268242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267883    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-lib-modules\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.268242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267955    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bad5cfd-a364-42c1-90dd-e2f7ddc39e64-host\") pod \"node-ca-7pdxl\" (UID: \"4bad5cfd-a364-42c1-90dd-e2f7ddc39e64\") " pod="openshift-image-registry/node-ca-7pdxl"
Sep 05 06:25:54.268242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.267989    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-hostroot\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.268242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.268021    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-etc-selinux\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.268242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.268106    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bad5cfd-a364-42c1-90dd-e2f7ddc39e64-host\") pod \"node-ca-7pdxl\" (UID: \"4bad5cfd-a364-42c1-90dd-e2f7ddc39e64\") " pod="openshift-image-registry/node-ca-7pdxl"
Sep 05 06:25:54.268710 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.268691    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8cae7568-294a-4c0e-adf9-d9581668c54e-konnectivity-ca\") pod \"konnectivity-agent-8xv7j\" (UID: \"8cae7568-294a-4c0e-adf9-d9581668c54e\") " pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:25:54.268794 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.268773    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-hostroot\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.268860 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.268803    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-run-ovn-kubernetes\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.268860 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.268823    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-log-socket\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.268860 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.268843    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-socket-dir-parent\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.269083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.268919    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-run-k8s-cni-cncf-io\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.269083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.268974    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lncbm\" (UniqueName: \"kubernetes.io/projected/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-kube-api-access-lncbm\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.269083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269004    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac5d72db-04fc-4fbc-9db7-959c7458ad90-dbus\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:54.269241 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269171    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:54.269241 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269201    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-etc-openvswitch\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.269241 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269207    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-run-k8s-cni-cncf-io\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.269241 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269234    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-run-ovn\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.269454 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269257    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-run-ovn-kubernetes\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.269454 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269270    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.269454 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269438    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-etc-selinux\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.269610 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269475    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-sys-fs\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.269610 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269512    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jpp\" (UniqueName: \"kubernetes.io/projected/2471243c-2dff-4f8f-a174-8ddf1264dacf-kube-api-access-b2jpp\") pod \"iptables-alerter-mx24p\" (UID: \"2471243c-2dff-4f8f-a174-8ddf1264dacf\") " pod="openshift-network-operator/iptables-alerter-mx24p"
Sep 05 06:25:54.269610 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269514    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-run-ovn\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.269610 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269551    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-modprobe-d\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.269610 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269589    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-cni-binary-copy\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.269940 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269901    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-kubelet\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270011 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269987    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-tuned\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.270011 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269995    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-multus-socket-dir-parent\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.270109 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270024    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-node-log\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270109 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270054    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/774d1813-9ee9-43c7-b398-bdb63b7f7e86-system-cni-dir\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.270109 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270089    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/774d1813-9ee9-43c7-b398-bdb63b7f7e86-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.270256 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270122    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/774d1813-9ee9-43c7-b398-bdb63b7f7e86-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.270256 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.269909    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-sys-fs\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.270460 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270434    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-host\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.270580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270480    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-cni-binary-copy\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.270580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270486    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-run-netns\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270509    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-tmp\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.270580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270519    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-run-openvswitch\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270537    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-etc-openvswitch\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270555    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-var-lib-cni-multus\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270591    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-device-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270710    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-modprobe-d\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270764    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270807    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-kubelet\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270821    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-device-dir\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270839    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-systemd-units\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270849    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-host\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270865    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-run-systemd\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270864    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-node-log\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270905    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-run-systemd\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270914    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-systemd-units\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.270946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.270917    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/774d1813-9ee9-43c7-b398-bdb63b7f7e86-system-cni-dir\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.271588 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.271079    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/774d1813-9ee9-43c7-b398-bdb63b7f7e86-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.271588 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.271116    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-run-openvswitch\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.271588 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.271207    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:25:54.271588 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.271267    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02c25fef-4296-4319-ba54-3ad5d77df1ca-ovn-node-metrics-cert\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.271588 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.271281    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret podName:ac5d72db-04fc-4fbc-9db7-959c7458ad90 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:54.771259573 +0000 UTC m=+3.139306764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret") pod "global-pull-secret-syncer-cjd64" (UID: "ac5d72db-04fc-4fbc-9db7-959c7458ad90") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:25:54.271588 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.271342    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-etc-kubernetes\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.271588 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.271449    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-etc-kubernetes\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.271588 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.271542    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02c25fef-4296-4319-ba54-3ad5d77df1ca-host-run-netns\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.272201 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.272176    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac5d72db-04fc-4fbc-9db7-959c7458ad90-dbus\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:54.272266 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.272251    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-host-var-lib-cni-multus\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.272323 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.272292    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/774d1813-9ee9-43c7-b398-bdb63b7f7e86-cni-binary-copy\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.272746 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.272724    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8cae7568-294a-4c0e-adf9-d9581668c54e-agent-certs\") pod \"konnectivity-agent-8xv7j\" (UID: \"8cae7568-294a-4c0e-adf9-d9581668c54e\") " pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:25:54.274172 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.274137    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/774d1813-9ee9-43c7-b398-bdb63b7f7e86-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.274370 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.274346    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/774d1813-9ee9-43c7-b398-bdb63b7f7e86-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.275381 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.275353    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 06:25:54.275685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.275662    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-etc-tuned\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.276845 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.276811    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02c25fef-4296-4319-ba54-3ad5d77df1ca-ovn-node-metrics-cert\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.280915 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.280815    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmdqn\" (UniqueName: \"kubernetes.io/projected/4bad5cfd-a364-42c1-90dd-e2f7ddc39e64-kube-api-access-cmdqn\") pod \"node-ca-7pdxl\" (UID: \"4bad5cfd-a364-42c1-90dd-e2f7ddc39e64\") " pod="openshift-image-registry/node-ca-7pdxl"
Sep 05 06:25:54.281041 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.280994    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgppn\" (UniqueName: \"kubernetes.io/projected/02c25fef-4296-4319-ba54-3ad5d77df1ca-kube-api-access-bgppn\") pod \"ovnkube-node-gbmg6\" (UID: \"02c25fef-4296-4319-ba54-3ad5d77df1ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.281622 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.281598    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2jpp\" (UniqueName: \"kubernetes.io/projected/2471243c-2dff-4f8f-a174-8ddf1264dacf-kube-api-access-b2jpp\") pod \"iptables-alerter-mx24p\" (UID: \"2471243c-2dff-4f8f-a174-8ddf1264dacf\") " pod="openshift-network-operator/iptables-alerter-mx24p"
Sep 05 06:25:54.281907 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.281836    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx9lv\" (UniqueName: \"kubernetes.io/projected/a9ddcc1d-0c0b-479e-9c07-c0f4dd425131-kube-api-access-vx9lv\") pod \"multus-tznq4\" (UID: \"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131\") " pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.283679 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.283652    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgxzq\" (UniqueName: \"kubernetes.io/projected/774d1813-9ee9-43c7-b398-bdb63b7f7e86-kube-api-access-lgxzq\") pod \"multus-additional-cni-plugins-tdcbd\" (UID: \"774d1813-9ee9-43c7-b398-bdb63b7f7e86\") " pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.284186 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.284167    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qscsv\" (UniqueName: \"kubernetes.io/projected/0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab-kube-api-access-qscsv\") pod \"tuned-ggtx7\" (UID: \"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab\") " pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.284629 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.284596    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lncbm\" (UniqueName: \"kubernetes.io/projected/57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f-kube-api-access-lncbm\") pod \"aws-ebs-csi-driver-node-whfmz\" (UID: \"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.365206 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.365162    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nq8t2"
Sep 05 06:25:54.373242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.373218    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7pdxl"
Sep 05 06:25:54.382374 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.382348    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tznq4"
Sep 05 06:25:54.393463 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.393431    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mx24p"
Sep 05 06:25:54.402821 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.402793    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tdcbd"
Sep 05 06:25:54.411929 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.411898    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ggtx7"
Sep 05 06:25:54.419061 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.419034    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz"
Sep 05 06:25:54.429084 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.429051    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:25:54.434237 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.434214    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:25:54.675631 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.675538    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:25:54.675631 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.675615    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t47pn\" (UniqueName: \"kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn\") pod \"network-check-target-76rz9\" (UID: \"24c5e798-bf33-4f1d-8296-6c4bcff878d6\") " pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:25:54.675870 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.675692    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:25:54.675870 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.675724    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:25:54.675870 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.675743    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:25:54.675870 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.675757    2573 projected.go:196] Error preparing data for projected volume kube-api-access-t47pn for pod openshift-network-diagnostics/network-check-target-76rz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:25:54.675870 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.675771    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs podName:e75f6f78-89ec-47e6-a11c-910a4362182d nodeName:}" failed. No retries permitted until 2026-09-05 06:25:55.675750512 +0000 UTC m=+4.043797728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs") pod "network-metrics-daemon-75v4r" (UID: "e75f6f78-89ec-47e6-a11c-910a4362182d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:25:54.675870 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.675794    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn podName:24c5e798-bf33-4f1d-8296-6c4bcff878d6 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:55.675784318 +0000 UTC m=+4.043831510 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t47pn" (UniqueName: "kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn") pod "network-check-target-76rz9" (UID: "24c5e798-bf33-4f1d-8296-6c4bcff878d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:25:54.776242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:54.776203    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:54.776450 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.776381    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:25:54.776522 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:54.776450    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret podName:ac5d72db-04fc-4fbc-9db7-959c7458ad90 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:55.776434492 +0000 UTC m=+4.144481688 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret") pod "global-pull-secret-syncer-cjd64" (UID: "ac5d72db-04fc-4fbc-9db7-959c7458ad90") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:25:55.103554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:55.103463    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 06:20:54 +0000 UTC" deadline="2028-06-09 03:26:17.845540018 +0000 UTC"
Sep 05 06:25:55.103554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:55.103498    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15429h0m22.742045761s"
Sep 05 06:25:55.150150 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:55.150114    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:25:55.150346 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:55.150233    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:25:55.228083 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:55.228027    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7095477_5141_481a_8b27_64e4c36845cb.slice/crio-eaf188a58b8caf9f79bfe595ff1f1895bc8e580047acab0d8d4660d239ea917e WatchSource:0}: Error finding container eaf188a58b8caf9f79bfe595ff1f1895bc8e580047acab0d8d4660d239ea917e: Status 404 returned error can't find the container with id eaf188a58b8caf9f79bfe595ff1f1895bc8e580047acab0d8d4660d239ea917e
Sep 05 06:25:55.230378 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:55.230347    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2471243c_2dff_4f8f_a174_8ddf1264dacf.slice/crio-5f1b83f2c867affc2bbc2da9c6bec4ab7c876ce8215a613b5f1d022caff8c793 WatchSource:0}: Error finding container 5f1b83f2c867affc2bbc2da9c6bec4ab7c876ce8215a613b5f1d022caff8c793: Status 404 returned error can't find the container with id 5f1b83f2c867affc2bbc2da9c6bec4ab7c876ce8215a613b5f1d022caff8c793
Sep 05 06:25:55.233162 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:55.233138    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod774d1813_9ee9_43c7_b398_bdb63b7f7e86.slice/crio-638bfd51d7ef9bf79d8c5ab5feea94e52be402709bd013401ba3e16f55f2d37b WatchSource:0}: Error finding container 638bfd51d7ef9bf79d8c5ab5feea94e52be402709bd013401ba3e16f55f2d37b: Status 404 returned error can't find the container with id 638bfd51d7ef9bf79d8c5ab5feea94e52be402709bd013401ba3e16f55f2d37b
Sep 05 06:25:55.234276 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:55.234236    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bad5cfd_a364_42c1_90dd_e2f7ddc39e64.slice/crio-15aa0b8d47f84703e3935a6356a8f06fc045d0b031c010235ed7ba88e585be47 WatchSource:0}: Error finding container 15aa0b8d47f84703e3935a6356a8f06fc045d0b031c010235ed7ba88e585be47: Status 404 returned error can't find the container with id 15aa0b8d47f84703e3935a6356a8f06fc045d0b031c010235ed7ba88e585be47
Sep 05 06:25:55.235010 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:55.234957    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cae7568_294a_4c0e_adf9_d9581668c54e.slice/crio-a53a37e2f5aaaf67e0336c442b73747ac9cd8c3f551234023bb0a400a35ef89e WatchSource:0}: Error finding container a53a37e2f5aaaf67e0336c442b73747ac9cd8c3f551234023bb0a400a35ef89e: Status 404 returned error can't find the container with id a53a37e2f5aaaf67e0336c442b73747ac9cd8c3f551234023bb0a400a35ef89e
Sep 05 06:25:55.236584 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:55.236558    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c25fef_4296_4319_ba54_3ad5d77df1ca.slice/crio-c55a9f89a4adf0e4fb8ebc0b9f7cd76f9c79d0dbd8f13b34ebe8e193c7127eea WatchSource:0}: Error finding container c55a9f89a4adf0e4fb8ebc0b9f7cd76f9c79d0dbd8f13b34ebe8e193c7127eea: Status 404 returned error can't find the container with id c55a9f89a4adf0e4fb8ebc0b9f7cd76f9c79d0dbd8f13b34ebe8e193c7127eea
Sep 05 06:25:55.238834 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:55.238709    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ea04db_b4ba_40b6_a3b8_cc3d4c70d64f.slice/crio-7284ca1064462e80def9f247e8fbc1502e15f53710595ad8e554837e84db9d9d WatchSource:0}: Error finding container 7284ca1064462e80def9f247e8fbc1502e15f53710595ad8e554837e84db9d9d: Status 404 returned error can't find the container with id 7284ca1064462e80def9f247e8fbc1502e15f53710595ad8e554837e84db9d9d
Sep 05 06:25:55.241043 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:25:55.239441    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dc2e90f_daf3_4a6e_8b46_c9e6b124b8ab.slice/crio-a5019a0ca4f530d7c850a08054017bcbb9094376ba0e867a8d367b4edf437c3f WatchSource:0}: Error finding container a5019a0ca4f530d7c850a08054017bcbb9094376ba0e867a8d367b4edf437c3f: Status 404 returned error can't find the container with id a5019a0ca4f530d7c850a08054017bcbb9094376ba0e867a8d367b4edf437c3f
Sep 05 06:25:55.683196 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:55.682876    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:25:55.683389 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:55.683226    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:25:55.683389 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:55.683248    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t47pn\" (UniqueName: \"kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn\") pod \"network-check-target-76rz9\" (UID: \"24c5e798-bf33-4f1d-8296-6c4bcff878d6\") " pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:25:55.683389 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:55.683316    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs podName:e75f6f78-89ec-47e6-a11c-910a4362182d nodeName:}" failed. No retries permitted until 2026-09-05 06:25:57.683293095 +0000 UTC m=+6.051340299 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs") pod "network-metrics-daemon-75v4r" (UID: "e75f6f78-89ec-47e6-a11c-910a4362182d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:25:55.683601 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:55.683399    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:25:55.683601 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:55.683504    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:25:55.683601 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:55.683520    2573 projected.go:196] Error preparing data for projected volume kube-api-access-t47pn for pod openshift-network-diagnostics/network-check-target-76rz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:25:55.683601 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:55.683573    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn podName:24c5e798-bf33-4f1d-8296-6c4bcff878d6 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:57.683556447 +0000 UTC m=+6.051603640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t47pn" (UniqueName: "kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn") pod "network-check-target-76rz9" (UID: "24c5e798-bf33-4f1d-8296-6c4bcff878d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:25:55.784387 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:55.783890    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:55.784387 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:55.784112    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:25:55.784387 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:55.784178    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret podName:ac5d72db-04fc-4fbc-9db7-959c7458ad90 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:57.784160311 +0000 UTC m=+6.152207504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret") pod "global-pull-secret-syncer-cjd64" (UID: "ac5d72db-04fc-4fbc-9db7-959c7458ad90") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:25:56.104175 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.104041    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 06:20:54 +0000 UTC" deadline="2028-03-14 20:44:19.897953135 +0000 UTC"
Sep 05 06:25:56.104175 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.104090    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13358h18m23.793869108s"
Sep 05 06:25:56.160071 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.160033    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:56.160278 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:56.160249    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:25:56.160865 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.160840    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:25:56.161104 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:56.161079    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:25:56.197798 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.197757    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8xv7j" event={"ID":"8cae7568-294a-4c0e-adf9-d9581668c54e","Type":"ContainerStarted","Data":"a53a37e2f5aaaf67e0336c442b73747ac9cd8c3f551234023bb0a400a35ef89e"}
Sep 05 06:25:56.203462 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.203418    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nq8t2" event={"ID":"e7095477-5141-481a-8b27-64e4c36845cb","Type":"ContainerStarted","Data":"eaf188a58b8caf9f79bfe595ff1f1895bc8e580047acab0d8d4660d239ea917e"}
Sep 05 06:25:56.213486 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.213446    2573 generic.go:356] "Generic (PLEG): container finished" podID="d469df84f974ddd8b8974ad2aded20b7" containerID="cf8b3783bee3d7b0e0b1a04eff5ae9d1d4d8999cb7051fe542e18d94966de9a4" exitCode=0
Sep 05 06:25:56.213681 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.213586    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal" event={"ID":"d469df84f974ddd8b8974ad2aded20b7","Type":"ContainerDied","Data":"cf8b3783bee3d7b0e0b1a04eff5ae9d1d4d8999cb7051fe542e18d94966de9a4"}
Sep 05 06:25:56.217793 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.217762    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ggtx7" event={"ID":"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab","Type":"ContainerStarted","Data":"a5019a0ca4f530d7c850a08054017bcbb9094376ba0e867a8d367b4edf437c3f"}
Sep 05 06:25:56.225030 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.224994    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdcbd" event={"ID":"774d1813-9ee9-43c7-b398-bdb63b7f7e86","Type":"ContainerStarted","Data":"638bfd51d7ef9bf79d8c5ab5feea94e52be402709bd013401ba3e16f55f2d37b"}
Sep 05 06:25:56.236696 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.236611    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mx24p" event={"ID":"2471243c-2dff-4f8f-a174-8ddf1264dacf","Type":"ContainerStarted","Data":"5f1b83f2c867affc2bbc2da9c6bec4ab7c876ce8215a613b5f1d022caff8c793"}
Sep 05 06:25:56.254509 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.254467    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-174.ec2.internal" event={"ID":"ddc0eaee28eb4599cb47316bccad4632","Type":"ContainerStarted","Data":"a7cc8819d00cfbab12c70c40904a663453ce042526de334514025ab22fbdc7f8"}
Sep 05 06:25:56.260999 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.260964    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tznq4" event={"ID":"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131","Type":"ContainerStarted","Data":"f6587815981737c6037f44e28beddfe52db659930e90a8cb3022a3cbf55dc69b"}
Sep 05 06:25:56.275451 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.272103    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz" event={"ID":"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f","Type":"ContainerStarted","Data":"7284ca1064462e80def9f247e8fbc1502e15f53710595ad8e554837e84db9d9d"}
Sep 05 06:25:56.294826 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.294783    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6" event={"ID":"02c25fef-4296-4319-ba54-3ad5d77df1ca","Type":"ContainerStarted","Data":"c55a9f89a4adf0e4fb8ebc0b9f7cd76f9c79d0dbd8f13b34ebe8e193c7127eea"}
Sep 05 06:25:56.324691 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:56.324601    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7pdxl" event={"ID":"4bad5cfd-a364-42c1-90dd-e2f7ddc39e64","Type":"ContainerStarted","Data":"15aa0b8d47f84703e3935a6356a8f06fc045d0b031c010235ed7ba88e585be47"}
Sep 05 06:25:57.150399 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:57.150302    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:25:57.150888 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:57.150470    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:25:57.347032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:57.346997    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal" event={"ID":"d469df84f974ddd8b8974ad2aded20b7","Type":"ContainerStarted","Data":"043048be49cdeaa4f66e228c7ac3e4f9d82dfcb9019bc6a5d6a69fb50deac45f"}
Sep 05 06:25:57.368039 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:57.367972    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-174.ec2.internal" podStartSLOduration=4.367948587 podStartE2EDuration="4.367948587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:56.287229935 +0000 UTC m=+4.655277150" watchObservedRunningTime="2026-09-05 06:25:57.367948587 +0000 UTC m=+5.735995801"
Sep 05 06:25:57.699869 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:57.699816    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:25:57.700062 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:57.699919    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t47pn\" (UniqueName: \"kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn\") pod \"network-check-target-76rz9\" (UID: \"24c5e798-bf33-4f1d-8296-6c4bcff878d6\") " pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:25:57.700142 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:57.700118    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:25:57.700210 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:57.700141    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:25:57.700210 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:57.700156    2573 projected.go:196] Error preparing data for projected volume kube-api-access-t47pn for pod openshift-network-diagnostics/network-check-target-76rz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:25:57.700311 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:57.700224    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn podName:24c5e798-bf33-4f1d-8296-6c4bcff878d6 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:01.700203585 +0000 UTC m=+10.068250779 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t47pn" (UniqueName: "kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn") pod "network-check-target-76rz9" (UID: "24c5e798-bf33-4f1d-8296-6c4bcff878d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:25:57.700699 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:57.700668    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:25:57.700823 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:57.700731    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs podName:e75f6f78-89ec-47e6-a11c-910a4362182d nodeName:}" failed. No retries permitted until 2026-09-05 06:26:01.700714551 +0000 UTC m=+10.068761756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs") pod "network-metrics-daemon-75v4r" (UID: "e75f6f78-89ec-47e6-a11c-910a4362182d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:25:57.801259 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:57.801217    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:57.801469 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:57.801418    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:25:57.801545 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:57.801496    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret podName:ac5d72db-04fc-4fbc-9db7-959c7458ad90 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:01.801475952 +0000 UTC m=+10.169523157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret") pod "global-pull-secret-syncer-cjd64" (UID: "ac5d72db-04fc-4fbc-9db7-959c7458ad90") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:25:58.150585 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:58.149980    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:25:58.150585 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:58.150138    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:25:58.150585 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:58.150151    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:25:58.150585 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:58.150297    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:25:59.149877 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:25:59.149677    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:25:59.149877 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:25:59.149819    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:00.151094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:00.150419    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:00.151094 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:00.150558    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:00.151094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:00.150419    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:00.151094 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:00.151051    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:01.149954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:01.149803    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:01.150104 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:01.149950    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:01.738061 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:01.738027    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:01.738502 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:01.738087    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t47pn\" (UniqueName: \"kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn\") pod \"network-check-target-76rz9\" (UID: \"24c5e798-bf33-4f1d-8296-6c4bcff878d6\") " pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:01.738502 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:01.738211    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:01.738502 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:01.738233    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:26:01.738502 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:01.738250    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:26:01.738502 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:01.738263    2573 projected.go:196] Error preparing data for projected volume kube-api-access-t47pn for pod openshift-network-diagnostics/network-check-target-76rz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:01.738502 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:01.738289    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs podName:e75f6f78-89ec-47e6-a11c-910a4362182d nodeName:}" failed. No retries permitted until 2026-09-05 06:26:09.738268551 +0000 UTC m=+18.106315752 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs") pod "network-metrics-daemon-75v4r" (UID: "e75f6f78-89ec-47e6-a11c-910a4362182d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:01.738502 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:01.738315    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn podName:24c5e798-bf33-4f1d-8296-6c4bcff878d6 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:09.738304245 +0000 UTC m=+18.106351448 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t47pn" (UniqueName: "kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn") pod "network-check-target-76rz9" (UID: "24c5e798-bf33-4f1d-8296-6c4bcff878d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:01.839460 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:01.839419    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:01.839639 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:01.839549    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:01.839639 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:01.839616    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret podName:ac5d72db-04fc-4fbc-9db7-959c7458ad90 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:09.839597708 +0000 UTC m=+18.207644897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret") pod "global-pull-secret-syncer-cjd64" (UID: "ac5d72db-04fc-4fbc-9db7-959c7458ad90") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:02.150261 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:02.150144    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:02.151187 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:02.151151    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:02.151719 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:02.151481    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:02.151719 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:02.151573    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:03.150109 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:03.150073    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:03.150589 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:03.150197    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:04.149707 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:04.149672    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:04.149903 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:04.149812    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:04.149903 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:04.149879    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:04.150040 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:04.150010    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:05.149535 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:05.149499    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:05.150027 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:05.149619    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:06.149951 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:06.149845    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:06.150361 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:06.149980    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:06.150361 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:06.150042    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:06.150361 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:06.150171    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:07.149999 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:07.149958    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:07.150423 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:07.150075    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:08.150202 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:08.150166    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:08.150652 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:08.150176    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:08.150652 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:08.150303    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:08.150652 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:08.150391    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:09.149334 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:09.149300    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:09.149572 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:09.149424    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:09.799729 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:09.799695    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:09.800192 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:09.799751    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t47pn\" (UniqueName: \"kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn\") pod \"network-check-target-76rz9\" (UID: \"24c5e798-bf33-4f1d-8296-6c4bcff878d6\") " pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:09.800192 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:09.799875    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:09.800192 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:09.799896    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:26:09.800192 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:09.799916    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:26:09.800192 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:09.799931    2573 projected.go:196] Error preparing data for projected volume kube-api-access-t47pn for pod openshift-network-diagnostics/network-check-target-76rz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:09.800192 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:09.799944    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs podName:e75f6f78-89ec-47e6-a11c-910a4362182d nodeName:}" failed. No retries permitted until 2026-09-05 06:26:25.799927255 +0000 UTC m=+34.167974459 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs") pod "network-metrics-daemon-75v4r" (UID: "e75f6f78-89ec-47e6-a11c-910a4362182d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:09.800192 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:09.799980    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn podName:24c5e798-bf33-4f1d-8296-6c4bcff878d6 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:25.799966425 +0000 UTC m=+34.168013630 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t47pn" (UniqueName: "kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn") pod "network-check-target-76rz9" (UID: "24c5e798-bf33-4f1d-8296-6c4bcff878d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:09.900361 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:09.900320    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:09.900544 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:09.900449    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:09.900544 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:09.900516    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret podName:ac5d72db-04fc-4fbc-9db7-959c7458ad90 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:25.90049935 +0000 UTC m=+34.268546544 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret") pod "global-pull-secret-syncer-cjd64" (UID: "ac5d72db-04fc-4fbc-9db7-959c7458ad90") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:10.149527 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:10.149446    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:10.149681 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:10.149446    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:10.149681 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:10.149586    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:10.149681 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:10.149665    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:11.150268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:11.150228    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:11.150731 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:11.150352    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:12.152108 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:12.152078    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:12.152108 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:12.152094    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:12.152532 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:12.152175    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:12.152532 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:12.152305    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:13.149639 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:13.149603    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:13.149811 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:13.149705    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:14.150514 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.150138    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:14.151180 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.150138    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:14.151180 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:14.150593    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:14.151180 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:14.150652    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:14.408045 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.407947    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz" event={"ID":"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f","Type":"ContainerStarted","Data":"c9a926962701f11144c6128661747bd3742118a310030042de96593772a51da0"}
Sep 05 06:26:14.411372 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.411331    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6" event={"ID":"02c25fef-4296-4319-ba54-3ad5d77df1ca","Type":"ContainerStarted","Data":"3e8466a07f6108c859e283d554f15bdb23fb4fce462e16de06d339ed47b65526"}
Sep 05 06:26:14.411534 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.411426    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6" event={"ID":"02c25fef-4296-4319-ba54-3ad5d77df1ca","Type":"ContainerStarted","Data":"8a9126deec4e274f492b7e56eddd2101acdbaffa9c9c905ca6b4fa7ca25e30a9"}
Sep 05 06:26:14.411534 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.411479    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6" event={"ID":"02c25fef-4296-4319-ba54-3ad5d77df1ca","Type":"ContainerStarted","Data":"35f037aa064c2ea939c883eec7641ef6032f100a6f4ade4d5a916537eb729988"}
Sep 05 06:26:14.411534 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.411532    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6" event={"ID":"02c25fef-4296-4319-ba54-3ad5d77df1ca","Type":"ContainerStarted","Data":"7c03b26a75364a37fcb2f2eda2ced2bdf29d77a68cdefd05e85b1023256f5de6"}
Sep 05 06:26:14.411720 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.411579    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6" event={"ID":"02c25fef-4296-4319-ba54-3ad5d77df1ca","Type":"ContainerStarted","Data":"3f45b8f6e37cad2652fcf249ac321a82fa7107e6971edeee300e5d87aa6d03e0"}
Sep 05 06:26:14.411720 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.411626    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6" event={"ID":"02c25fef-4296-4319-ba54-3ad5d77df1ca","Type":"ContainerStarted","Data":"99f35059a0580e4b3262e98082f97df1dd0e8f6f6f5a946f23a283d76c2d3157"}
Sep 05 06:26:14.412998 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.412961    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7pdxl" event={"ID":"4bad5cfd-a364-42c1-90dd-e2f7ddc39e64","Type":"ContainerStarted","Data":"1d4f77f1748520b9ff7e87b2dce7d2fa5971d63db360f8653ac583a0f13b2583"}
Sep 05 06:26:14.415000 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.414639    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8xv7j" event={"ID":"8cae7568-294a-4c0e-adf9-d9581668c54e","Type":"ContainerStarted","Data":"d89b380cea28a5879363f30d14c8a2ded3d9d39919d0bbfea5754aa3d7006121"}
Sep 05 06:26:14.416242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.416217    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nq8t2" event={"ID":"e7095477-5141-481a-8b27-64e4c36845cb","Type":"ContainerStarted","Data":"753f2299d342ec17beae6457628629e061bddf64000662ce897271bbee4deaf6"}
Sep 05 06:26:14.418018 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.417988    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ggtx7" event={"ID":"0dc2e90f-daf3-4a6e-8b46-c9e6b124b8ab","Type":"ContainerStarted","Data":"fa736ededa5b4869e4414c9a6afb17b10a4199341c755fa22ceb4ac177b03959"}
Sep 05 06:26:14.419697 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.419675    2573 generic.go:356] "Generic (PLEG): container finished" podID="774d1813-9ee9-43c7-b398-bdb63b7f7e86" containerID="30a186c2b5ebf2c3f422cf7db1d5c72c1c6fe923402c1979771b1e035f1f9502" exitCode=0
Sep 05 06:26:14.419781 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.419749    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdcbd" event={"ID":"774d1813-9ee9-43c7-b398-bdb63b7f7e86","Type":"ContainerDied","Data":"30a186c2b5ebf2c3f422cf7db1d5c72c1c6fe923402c1979771b1e035f1f9502"}
Sep 05 06:26:14.421283 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.421260    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tznq4" event={"ID":"a9ddcc1d-0c0b-479e-9c07-c0f4dd425131","Type":"ContainerStarted","Data":"bbc0c52e704fd340be45cbcbb2cbe613b5aa8080de4b21da62e7bc9d7208d935"}
Sep 05 06:26:14.430885 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.430822    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-174.ec2.internal" podStartSLOduration=21.430805455 podStartE2EDuration="21.430805455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:57.369181643 +0000 UTC m=+5.737228857" watchObservedRunningTime="2026-09-05 06:26:14.430805455 +0000 UTC m=+22.798852665"
Sep 05 06:26:14.431229 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.431199    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7pdxl" podStartSLOduration=4.395572738 podStartE2EDuration="22.431192601s" totalImagesPullingTime="18.035619863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:14.430885237 +0000 UTC m=+22.798932448" watchObservedRunningTime="2026-09-05 06:26:14.431192601 +0000 UTC m=+22.799239841"
Sep 05 06:26:14.451269 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.451209    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nq8t2" podStartSLOduration=4.406979222 podStartE2EDuration="22.451188461s" totalImagesPullingTime="18.044209239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:14.450807056 +0000 UTC m=+22.818854269" watchObservedRunningTime="2026-09-05 06:26:14.451188461 +0000 UTC m=+22.819235672"
Sep 05 06:26:14.471448 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.471387    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tznq4" podStartSLOduration=4.377856222 podStartE2EDuration="22.471368171s" totalImagesPullingTime="18.093511949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:14.471175502 +0000 UTC m=+22.839222708" watchObservedRunningTime="2026-09-05 06:26:14.471368171 +0000 UTC m=+22.839415381"
Sep 05 06:26:14.486329 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.486282    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8xv7j" podStartSLOduration=10.808191268 podStartE2EDuration="21.486262101s" totalImagesPullingTime="10.678070833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:14.485835494 +0000 UTC m=+22.853882705" watchObservedRunningTime="2026-09-05 06:26:14.486262101 +0000 UTC m=+22.854309312"
Sep 05 06:26:14.525705 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.525653    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ggtx7" podStartSLOduration=4.491476802 podStartE2EDuration="22.525636584s" totalImagesPullingTime="18.034159782s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:14.52505582 +0000 UTC m=+22.893103029" watchObservedRunningTime="2026-09-05 06:26:14.525636584 +0000 UTC m=+22.893683825"
Sep 05 06:26:14.927619 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:14.927588    2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 05 06:26:15.132124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:15.131907    2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-05T06:26:14.927612718Z","UUID":"80cc91ea-e02f-4cfe-800d-399e4724fd34","Handler":null,"Name":"","Endpoint":""}
Sep 05 06:26:15.133724 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:15.133692    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 05 06:26:15.133724 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:15.133728    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 05 06:26:15.150286 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:15.150258    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:15.150489 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:15.150370    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:15.426060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:15.425973    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mx24p" event={"ID":"2471243c-2dff-4f8f-a174-8ddf1264dacf","Type":"ContainerStarted","Data":"940b1a9470fa36fd973885b0dd949a845b32f9addb5c55f4556351f7885f0b6c"}
Sep 05 06:26:15.428058 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:15.428027    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz" event={"ID":"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f","Type":"ContainerStarted","Data":"5b0e7c18aa4b8977281b0e8d39dd84e1cd0195b64f64b9ef803de135be016f0b"}
Sep 05 06:26:15.445283 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:15.445225    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mx24p" podStartSLOduration=5.403054601 podStartE2EDuration="23.445205122s" totalImagesPullingTime="18.042150521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:15.444979139 +0000 UTC m=+23.813026348" watchObservedRunningTime="2026-09-05 06:26:15.445205122 +0000 UTC m=+23.813252336"
Sep 05 06:26:15.569773 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:15.569716    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:26:15.570456 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:15.570431    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:26:16.149894 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:16.149862    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:16.150047 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:16.150014    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:16.150110 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:16.150062    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:16.150218 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:16.150191    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:16.433221 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:16.433129    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz" event={"ID":"57ea04db-b4ba-40b6-a3b8-cc3d4c70d64f","Type":"ContainerStarted","Data":"80c6c3a9d93ba1c35c24b2e8d55646be19641eb2f170b0d7b0fb8fa1aa0f68b6"}
Sep 05 06:26:16.436954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:16.436925    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6" event={"ID":"02c25fef-4296-4319-ba54-3ad5d77df1ca","Type":"ContainerStarted","Data":"8f8296f732849d8ecf1c4281fdbf897038fac29d9b7501d4ec8bd3f59a707f6a"}
Sep 05 06:26:16.437394 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:16.437375    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:26:16.437884 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:16.437863    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8xv7j"
Sep 05 06:26:16.455843 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:16.455798    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whfmz" podStartSLOduration=3.996581118 podStartE2EDuration="24.455780663s" totalImagesPullingTime="20.459199545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:16.455567582 +0000 UTC m=+24.823614793" watchObservedRunningTime="2026-09-05 06:26:16.455780663 +0000 UTC m=+24.823827870"
Sep 05 06:26:17.149916 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:17.149883    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:17.150112 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:17.150011    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:18.149708 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:18.149670    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:18.150183 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:18.149726    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:18.150183 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:18.149817    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:18.150183 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:18.149957    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:19.150376 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:19.150287    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:19.151235 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:19.150405    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:19.447167 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:19.447071    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6" event={"ID":"02c25fef-4296-4319-ba54-3ad5d77df1ca","Type":"ContainerStarted","Data":"152b3481c3fd89371453ea427c4611912cabdeee53853696c02ee339007939f4"}
Sep 05 06:26:19.447394 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:19.447366    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:26:19.447470 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:19.447446    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:26:19.449110 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:19.449084    2573 generic.go:356] "Generic (PLEG): container finished" podID="774d1813-9ee9-43c7-b398-bdb63b7f7e86" containerID="d9be8b87928a525a5efc8e4e9836813a1b64d65dd72f76d2792b28f7c80579e8" exitCode=0
Sep 05 06:26:19.449228 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:19.449125    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdcbd" event={"ID":"774d1813-9ee9-43c7-b398-bdb63b7f7e86","Type":"ContainerDied","Data":"d9be8b87928a525a5efc8e4e9836813a1b64d65dd72f76d2792b28f7c80579e8"}
Sep 05 06:26:19.463254 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:19.463219    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:26:19.481597 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:19.481547    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6" podStartSLOduration=8.403253452 podStartE2EDuration="26.481532574s" totalImagesPullingTime="18.078279122s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:19.480203392 +0000 UTC m=+27.848250597" watchObservedRunningTime="2026-09-05 06:26:19.481532574 +0000 UTC m=+27.849579783"
Sep 05 06:26:20.150255 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:20.150074    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:20.150489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:20.150087    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:20.150489 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:20.150367    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:20.150489 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:20.150436    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:20.452488 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:20.452452    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:26:20.470146 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:20.470119    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:26:20.741341 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:20.741257    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cjd64"]
Sep 05 06:26:20.741507 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:20.741410    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:20.741568 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:20.741516    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:20.744687 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:20.744661    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-76rz9"]
Sep 05 06:26:20.744814 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:20.744788    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:20.744918 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:20.744899    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:20.747019 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:20.746993    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-75v4r"]
Sep 05 06:26:20.747117 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:20.747086    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:20.747177 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:20.747160    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:21.455837 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:21.455787    2573 generic.go:356] "Generic (PLEG): container finished" podID="774d1813-9ee9-43c7-b398-bdb63b7f7e86" containerID="f1c8067b61f6f3bbc62d7cbccaa9fe7df46bbcb5f2a67a37d53290fe569808ec" exitCode=0
Sep 05 06:26:21.456330 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:21.455880    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdcbd" event={"ID":"774d1813-9ee9-43c7-b398-bdb63b7f7e86","Type":"ContainerDied","Data":"f1c8067b61f6f3bbc62d7cbccaa9fe7df46bbcb5f2a67a37d53290fe569808ec"}
Sep 05 06:26:22.150530 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:22.150489    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:22.150694 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:22.150577    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:22.150694 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:22.150613    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:22.150794 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:22.150709    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:22.150794 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:22.150755    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:22.150910 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:22.150824    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:22.460802 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:22.460721    2573 generic.go:356] "Generic (PLEG): container finished" podID="774d1813-9ee9-43c7-b398-bdb63b7f7e86" containerID="7f28a25ae74bfbb961edd5254b2b73c1e57a3bccc448675c295acc44ae759eb4" exitCode=0
Sep 05 06:26:22.460802 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:22.460785    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdcbd" event={"ID":"774d1813-9ee9-43c7-b398-bdb63b7f7e86","Type":"ContainerDied","Data":"7f28a25ae74bfbb961edd5254b2b73c1e57a3bccc448675c295acc44ae759eb4"}
Sep 05 06:26:24.149627 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:24.149572    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:24.150101 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:24.149575    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:24.150101 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:24.149716    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:24.150101 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:24.149714    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-76rz9" podUID="24c5e798-bf33-4f1d-8296-6c4bcff878d6"
Sep 05 06:26:24.150101 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:24.149799    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjd64" podUID="ac5d72db-04fc-4fbc-9db7-959c7458ad90"
Sep 05 06:26:24.150101 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:24.149900    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-75v4r" podUID="e75f6f78-89ec-47e6-a11c-910a4362182d"
Sep 05 06:26:25.820373 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:25.820186    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:25.820793 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:25.820372    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:25.820793 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:25.820417    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t47pn\" (UniqueName: \"kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn\") pod \"network-check-target-76rz9\" (UID: \"24c5e798-bf33-4f1d-8296-6c4bcff878d6\") " pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:25.820793 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:25.820454    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs podName:e75f6f78-89ec-47e6-a11c-910a4362182d nodeName:}" failed. No retries permitted until 2026-09-05 06:26:57.820433722 +0000 UTC m=+66.188480931 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs") pod "network-metrics-daemon-75v4r" (UID: "e75f6f78-89ec-47e6-a11c-910a4362182d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:25.820793 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:25.820535    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:26:25.820793 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:25.820555    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:26:25.820793 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:25.820568    2573 projected.go:196] Error preparing data for projected volume kube-api-access-t47pn for pod openshift-network-diagnostics/network-check-target-76rz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:25.820793 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:25.820616    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn podName:24c5e798-bf33-4f1d-8296-6c4bcff878d6 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:57.820599916 +0000 UTC m=+66.188647121 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-t47pn" (UniqueName: "kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn") pod "network-check-target-76rz9" (UID: "24c5e798-bf33-4f1d-8296-6c4bcff878d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:25.921255 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:25.921213    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:25.921452 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:25.921381    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:25.921452 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:25.921452    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret podName:ac5d72db-04fc-4fbc-9db7-959c7458ad90 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:57.921436814 +0000 UTC m=+66.289484001 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret") pod "global-pull-secret-syncer-cjd64" (UID: "ac5d72db-04fc-4fbc-9db7-959c7458ad90") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:25.928045 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:25.928016    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-174.ec2.internal" event="NodeReady"
Sep 05 06:26:25.928196 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:25.928174    2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 05 06:26:25.972653 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:25.972618    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6bc8d97d45-s96zd"]
Sep 05 06:26:26.006736 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.006701    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xpm9h"]
Sep 05 06:26:26.006927 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.006836    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.011013 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.010987    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 05 06:26:26.011157 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.011117    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 05 06:26:26.011491 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.011472    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rfhpd\""
Sep 05 06:26:26.014486 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.014316    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 05 06:26:26.018089 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.018068    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 05 06:26:26.018613 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.018591    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bc8d97d45-s96zd"]
Sep 05 06:26:26.018725 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.018630    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-llbmp"]
Sep 05 06:26:26.018725 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.018702    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.021239 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.021220    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 05 06:26:26.021381 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.021364    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v76tl\""
Sep 05 06:26:26.021452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.021364    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 05 06:26:26.041367 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.041341    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xpm9h"]
Sep 05 06:26:26.041502 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.041387    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-llbmp"]
Sep 05 06:26:26.041502 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.041479    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-llbmp"
Sep 05 06:26:26.045057 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.045035    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 05 06:26:26.045186 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.045131    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 05 06:26:26.045186 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.045145    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 05 06:26:26.045408 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.045391    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qnm9m\""
Sep 05 06:26:26.122661 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.122575    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae33df4f-832d-4ee1-83b7-798f60f540d8-config-volume\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.122661 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.122624    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-trusted-ca\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.122661 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.122653    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae33df4f-832d-4ee1-83b7-798f60f540d8-tmp-dir\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.122923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.122681    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbf8d038-21ba-42eb-aac4-7b16c7eae743-ca-trust-extracted\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.122923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.122707    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.122923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.122765    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nzpp\" (UniqueName: \"kubernetes.io/projected/5d2c676f-9881-4180-9aa4-2cc3757adfa0-kube-api-access-5nzpp\") pod \"ingress-canary-llbmp\" (UID: \"5d2c676f-9881-4180-9aa4-2cc3757adfa0\") " pod="openshift-ingress-canary/ingress-canary-llbmp"
Sep 05 06:26:26.122923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.122841    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnfnf\" (UniqueName: \"kubernetes.io/projected/ae33df4f-832d-4ee1-83b7-798f60f540d8-kube-api-access-xnfnf\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.122923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.122890    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-image-registry-private-configuration\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.123134 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.122926    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d2c676f-9881-4180-9aa4-2cc3757adfa0-cert\") pod \"ingress-canary-llbmp\" (UID: \"5d2c676f-9881-4180-9aa4-2cc3757adfa0\") " pod="openshift-ingress-canary/ingress-canary-llbmp"
Sep 05 06:26:26.123134 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.122998    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.123134 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.123026    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-certificates\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.123134 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.123076    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-installation-pull-secrets\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.123134 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.123109    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mwp7\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-kube-api-access-2mwp7\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.123315 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.123154    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-bound-sa-token\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.150319 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.150282    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:26.150488 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.150324    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:26.150488 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.150291    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:26.156768 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.156741    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 05 06:26:26.157034 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.157011    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 05 06:26:26.157193 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.157178    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 05 06:26:26.157267 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.157223    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4m5kh\""
Sep 05 06:26:26.157357 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.157343    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 05 06:26:26.157590 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.157567    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-24sqj\""
Sep 05 06:26:26.172188 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.172137    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:26.223714 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.223678    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-certificates\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.223937 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.223736    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-installation-pull-secrets\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.223937 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.223764    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mwp7\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-kube-api-access-2mwp7\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.223937 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.223791    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-bound-sa-token\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.223937 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.223807    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae33df4f-832d-4ee1-83b7-798f60f540d8-config-volume\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.224151 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.224017    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-trusted-ca\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.224463 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.224439    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-certificates\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.224834 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.224798    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ae33df4f-832d-4ee1-83b7-798f60f540d8-config-volume\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.224940 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.224844    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae33df4f-832d-4ee1-83b7-798f60f540d8-tmp-dir\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.224940 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.224046    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae33df4f-832d-4ee1-83b7-798f60f540d8-tmp-dir\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.224940 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.224924    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbf8d038-21ba-42eb-aac4-7b16c7eae743-ca-trust-extracted\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.225082 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.224951    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.225082 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.224977    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nzpp\" (UniqueName: \"kubernetes.io/projected/5d2c676f-9881-4180-9aa4-2cc3757adfa0-kube-api-access-5nzpp\") pod \"ingress-canary-llbmp\" (UID: \"5d2c676f-9881-4180-9aa4-2cc3757adfa0\") " pod="openshift-ingress-canary/ingress-canary-llbmp"
Sep 05 06:26:26.225082 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.225011    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnfnf\" (UniqueName: \"kubernetes.io/projected/ae33df4f-832d-4ee1-83b7-798f60f540d8-kube-api-access-xnfnf\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.225082 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.225044    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-image-registry-private-configuration\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.225082 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.225074    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d2c676f-9881-4180-9aa4-2cc3757adfa0-cert\") pod \"ingress-canary-llbmp\" (UID: \"5d2c676f-9881-4180-9aa4-2cc3757adfa0\") " pod="openshift-ingress-canary/ingress-canary-llbmp"
Sep 05 06:26:26.225292 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.225125    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.225292 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:26.225241    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 06:26:26.225292 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:26.225257    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc8d97d45-s96zd: secret "image-registry-tls" not found
Sep 05 06:26:26.225407 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:26.225307    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls podName:dbf8d038-21ba-42eb-aac4-7b16c7eae743 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:26.725290966 +0000 UTC m=+35.093338158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls") pod "image-registry-6bc8d97d45-s96zd" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743") : secret "image-registry-tls" not found
Sep 05 06:26:26.225407 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.225372    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-trusted-ca\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.225578 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.225555    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbf8d038-21ba-42eb-aac4-7b16c7eae743-ca-trust-extracted\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.225693 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:26.225667    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:26:26.225757 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:26.225723    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls podName:ae33df4f-832d-4ee1-83b7-798f60f540d8 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:26.725708019 +0000 UTC m=+35.093755224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls") pod "dns-default-xpm9h" (UID: "ae33df4f-832d-4ee1-83b7-798f60f540d8") : secret "dns-default-metrics-tls" not found
Sep 05 06:26:26.228903 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.228879    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-installation-pull-secrets\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.228903 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.228901    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d2c676f-9881-4180-9aa4-2cc3757adfa0-cert\") pod \"ingress-canary-llbmp\" (UID: \"5d2c676f-9881-4180-9aa4-2cc3757adfa0\") " pod="openshift-ingress-canary/ingress-canary-llbmp"
Sep 05 06:26:26.229071 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.228879    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-image-registry-private-configuration\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.241102 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.241078    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mwp7\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-kube-api-access-2mwp7\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.247964 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.247938    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-bound-sa-token\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.248726 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.248704    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnfnf\" (UniqueName: \"kubernetes.io/projected/ae33df4f-832d-4ee1-83b7-798f60f540d8-kube-api-access-xnfnf\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.249216 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.249193    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nzpp\" (UniqueName: \"kubernetes.io/projected/5d2c676f-9881-4180-9aa4-2cc3757adfa0-kube-api-access-5nzpp\") pod \"ingress-canary-llbmp\" (UID: \"5d2c676f-9881-4180-9aa4-2cc3757adfa0\") " pod="openshift-ingress-canary/ingress-canary-llbmp"
Sep 05 06:26:26.282408 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.282369    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:26.350440 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.350404    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-llbmp"
Sep 05 06:26:26.554586 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.554549    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:26.639269 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.639177    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:26.729347 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.729313    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:26.729519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.729361    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:26.729519 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:26.729460    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 06:26:26.729519 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:26.729473    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc8d97d45-s96zd: secret "image-registry-tls" not found
Sep 05 06:26:26.729519 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:26.729515    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls podName:dbf8d038-21ba-42eb-aac4-7b16c7eae743 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:27.729503174 +0000 UTC m=+36.097550366 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls") pod "image-registry-6bc8d97d45-s96zd" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743") : secret "image-registry-tls" not found
Sep 05 06:26:26.729668 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:26.729460    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:26:26.729668 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:26.729544    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls podName:ae33df4f-832d-4ee1-83b7-798f60f540d8 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:27.729538858 +0000 UTC m=+36.097586047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls") pod "dns-default-xpm9h" (UID: "ae33df4f-832d-4ee1-83b7-798f60f540d8") : secret "dns-default-metrics-tls" not found
Sep 05 06:26:26.892217 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:26.892157    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-llbmp"]
Sep 05 06:26:27.475010 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:27.474974    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-llbmp" event={"ID":"5d2c676f-9881-4180-9aa4-2cc3757adfa0","Type":"ContainerStarted","Data":"a14286fb9c602b2fb0658a1e42ad238a2c4125576f52d7af44c8be417de5af29"}
Sep 05 06:26:27.478188 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:27.478153    2573 generic.go:356] "Generic (PLEG): container finished" podID="774d1813-9ee9-43c7-b398-bdb63b7f7e86" containerID="c9c1d69654244acdab82c2e80424fbe6f9a1c434d7289b4a00027dbf95a27707" exitCode=0
Sep 05 06:26:27.478312 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:27.478214    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdcbd" event={"ID":"774d1813-9ee9-43c7-b398-bdb63b7f7e86","Type":"ContainerDied","Data":"c9c1d69654244acdab82c2e80424fbe6f9a1c434d7289b4a00027dbf95a27707"}
Sep 05 06:26:27.738986 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:27.738911    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:27.738986 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:27.738977    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:27.739213 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:27.739088    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:26:27.739213 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:27.739111    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 06:26:27.739213 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:27.739125    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc8d97d45-s96zd: secret "image-registry-tls" not found
Sep 05 06:26:27.739213 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:27.739165    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls podName:ae33df4f-832d-4ee1-83b7-798f60f540d8 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:29.73914424 +0000 UTC m=+38.107191439 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls") pod "dns-default-xpm9h" (UID: "ae33df4f-832d-4ee1-83b7-798f60f540d8") : secret "dns-default-metrics-tls" not found
Sep 05 06:26:27.739213 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:27.739186    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls podName:dbf8d038-21ba-42eb-aac4-7b16c7eae743 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:29.739176156 +0000 UTC m=+38.107223350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls") pod "image-registry-6bc8d97d45-s96zd" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743") : secret "image-registry-tls" not found
Sep 05 06:26:28.483367 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:28.483338    2573 generic.go:356] "Generic (PLEG): container finished" podID="774d1813-9ee9-43c7-b398-bdb63b7f7e86" containerID="beb32d02531d1265fc33c4d96a7a16fa868e10e4eb0953223e9634f473cda55f" exitCode=0
Sep 05 06:26:28.483869 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:28.483379    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdcbd" event={"ID":"774d1813-9ee9-43c7-b398-bdb63b7f7e86","Type":"ContainerDied","Data":"beb32d02531d1265fc33c4d96a7a16fa868e10e4eb0953223e9634f473cda55f"}
Sep 05 06:26:29.490045 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:29.489785    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdcbd" event={"ID":"774d1813-9ee9-43c7-b398-bdb63b7f7e86","Type":"ContainerStarted","Data":"15efdea0f40181ec3f9378a13d73d3e13fdce80c36607d89e6dc78926fc7588a"}
Sep 05 06:26:29.517016 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:29.516922    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tdcbd" podStartSLOduration=9.02520289 podStartE2EDuration="37.516905311s" totalImagesPullingTime="28.491702421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:29.516788854 +0000 UTC m=+37.884836063" watchObservedRunningTime="2026-09-05 06:26:29.516905311 +0000 UTC m=+37.884952520"
Sep 05 06:26:29.670411 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:29.670367    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:29.758609 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:29.758569    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:29.758784 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:29.758717    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:26:29.758784 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:29.758750    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:29.758784 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:29.758775    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls podName:ae33df4f-832d-4ee1-83b7-798f60f540d8 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:33.758760798 +0000 UTC m=+42.126807990 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls") pod "dns-default-xpm9h" (UID: "ae33df4f-832d-4ee1-83b7-798f60f540d8") : secret "dns-default-metrics-tls" not found
Sep 05 06:26:29.758921 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:29.758827    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 06:26:29.758921 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:29.758837    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc8d97d45-s96zd: secret "image-registry-tls" not found
Sep 05 06:26:29.758921 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:29.758895    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls podName:dbf8d038-21ba-42eb-aac4-7b16c7eae743 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:33.758883863 +0000 UTC m=+42.126931052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls") pod "image-registry-6bc8d97d45-s96zd" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743") : secret "image-registry-tls" not found
Sep 05 06:26:29.787948 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:29.787845    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:30.493644 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:30.493611    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-llbmp" event={"ID":"5d2c676f-9881-4180-9aa4-2cc3757adfa0","Type":"ContainerStarted","Data":"414156dc2082395b71782f807c91e2243620680523192d11cdaf8ce679925f87"}
Sep 05 06:26:30.512201 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:30.512147    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-llbmp" podStartSLOduration=2.9293189330000002 podStartE2EDuration="5.512129346s" totalImagesPullingTime="2.582810413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:30.511599731 +0000 UTC m=+38.879646939" watchObservedRunningTime="2026-09-05 06:26:30.512129346 +0000 UTC m=+38.880176556"
Sep 05 06:26:31.650829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:31.650798    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:31.712608 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:31.712565    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:33.787605 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:33.787555    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:33.788011 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:33.787623    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:33.788011 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:33.787707    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:26:33.788011 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:33.787717    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 06:26:33.788011 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:33.787766    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc8d97d45-s96zd: secret "image-registry-tls" not found
Sep 05 06:26:33.788011 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:33.787771    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls podName:ae33df4f-832d-4ee1-83b7-798f60f540d8 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:41.787756557 +0000 UTC m=+50.155803750 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls") pod "dns-default-xpm9h" (UID: "ae33df4f-832d-4ee1-83b7-798f60f540d8") : secret "dns-default-metrics-tls" not found
Sep 05 06:26:33.788011 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:33.787807    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls podName:dbf8d038-21ba-42eb-aac4-7b16c7eae743 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:41.787792435 +0000 UTC m=+50.155839641 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls") pod "image-registry-6bc8d97d45-s96zd" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743") : secret "image-registry-tls" not found
Sep 05 06:26:38.537762 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:38.537725    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:38.639779 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:38.639746    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:38.702462 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:38.702430    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:39.461387 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:39.461353    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:40.574890 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:40.574841    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:40.676165 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:40.676128    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:41.844954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:41.844908    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:41.845350 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:41.845010    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:41.845350 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:41.845063    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 06:26:41.845350 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:41.845083    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc8d97d45-s96zd: secret "image-registry-tls" not found
Sep 05 06:26:41.845350 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:41.845120    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:26:41.845350 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:41.845160    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls podName:dbf8d038-21ba-42eb-aac4-7b16c7eae743 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:57.845139372 +0000 UTC m=+66.213186579 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls") pod "image-registry-6bc8d97d45-s96zd" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743") : secret "image-registry-tls" not found
Sep 05 06:26:41.845350 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:41.845177    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls podName:ae33df4f-832d-4ee1-83b7-798f60f540d8 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:57.845166183 +0000 UTC m=+66.213213371 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls") pod "dns-default-xpm9h" (UID: "ae33df4f-832d-4ee1-83b7-798f60f540d8") : secret "dns-default-metrics-tls" not found
Sep 05 06:26:46.205701 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:46.205658    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:46.275272 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:46.275241    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:46.375357 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:46.375333    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:46.438423 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:46.438393    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:47.344085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:47.344047    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:47.441845 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:47.441804    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:51.283808 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:51.283773    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:52.471555 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:52.471524    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbmg6"
Sep 05 06:26:52.741327 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:52.741290    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:52.806566 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:52.806532    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:54.630911 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:54.630875    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:54.730586 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:54.730533    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:55.911216 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:55.911180    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:56.012988 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:56.012956    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:56.996746 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:56.996712    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:57.055768 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.055736    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:57.856081 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.856028    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:26:57.856309 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.856097    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:26:57.856309 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.856134    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t47pn\" (UniqueName: \"kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn\") pod \"network-check-target-76rz9\" (UID: \"24c5e798-bf33-4f1d-8296-6c4bcff878d6\") " pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:57.856309 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.856165    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:26:57.856309 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:57.856242    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 06:26:57.856309 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:57.856254    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc8d97d45-s96zd: secret "image-registry-tls" not found
Sep 05 06:26:57.856309 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:57.856291    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:26:57.856309 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:57.856314    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls podName:dbf8d038-21ba-42eb-aac4-7b16c7eae743 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:29.856299091 +0000 UTC m=+98.224346279 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls") pod "image-registry-6bc8d97d45-s96zd" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743") : secret "image-registry-tls" not found
Sep 05 06:26:57.856571 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:57.856363    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls podName:ae33df4f-832d-4ee1-83b7-798f60f540d8 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:29.856343572 +0000 UTC m=+98.224390786 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls") pod "dns-default-xpm9h" (UID: "ae33df4f-832d-4ee1-83b7-798f60f540d8") : secret "dns-default-metrics-tls" not found
Sep 05 06:26:57.859503 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.859486    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 05 06:26:57.859609 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.859589    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 05 06:26:57.866814 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:57.866797    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 05 06:26:57.866875 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:26:57.866866    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs podName:e75f6f78-89ec-47e6-a11c-910a4362182d nodeName:}" failed. No retries permitted until 2026-09-05 06:28:01.86683649 +0000 UTC m=+130.234883683 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs") pod "network-metrics-daemon-75v4r" (UID: "e75f6f78-89ec-47e6-a11c-910a4362182d") : secret "metrics-daemon-secret" not found
Sep 05 06:26:57.868949 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.868934    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 05 06:26:57.881072 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.881040    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t47pn\" (UniqueName: \"kubernetes.io/projected/24c5e798-bf33-4f1d-8296-6c4bcff878d6-kube-api-access-t47pn\") pod \"network-check-target-76rz9\" (UID: \"24c5e798-bf33-4f1d-8296-6c4bcff878d6\") " pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:57.956906 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.956867    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:57.960078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.960057    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 05 06:26:57.969863 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.969818    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac5d72db-04fc-4fbc-9db7-959c7458ad90-original-pull-secret\") pod \"global-pull-secret-syncer-cjd64\" (UID: \"ac5d72db-04fc-4fbc-9db7-959c7458ad90\") " pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:57.976563 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.976546    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-24sqj\""
Sep 05 06:26:57.984886 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:57.984864    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:26:58.255925 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:58.255894    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-76rz9"]
Sep 05 06:26:58.259229 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:26:58.259204    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c5e798_bf33_4f1d_8296_6c4bcff878d6.slice/crio-826fca712a935cc7968c1097a92298f19baa52dddc53b2203dcea2569313ec19 WatchSource:0}: Error finding container 826fca712a935cc7968c1097a92298f19baa52dddc53b2203dcea2569313ec19: Status 404 returned error can't find the container with id 826fca712a935cc7968c1097a92298f19baa52dddc53b2203dcea2569313ec19
Sep 05 06:26:58.269543 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:58.269514    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjd64"
Sep 05 06:26:58.540992 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:58.540903    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cjd64"]
Sep 05 06:26:58.544482 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:26:58.544442    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5d72db_04fc_4fbc_9db7_959c7458ad90.slice/crio-8cb86e184ec462c833c66a60e9cdd99bd32dd3d5af84f57747d3a54225ff025b WatchSource:0}: Error finding container 8cb86e184ec462c833c66a60e9cdd99bd32dd3d5af84f57747d3a54225ff025b: Status 404 returned error can't find the container with id 8cb86e184ec462c833c66a60e9cdd99bd32dd3d5af84f57747d3a54225ff025b
Sep 05 06:26:58.561831 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:58.561800    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cjd64" event={"ID":"ac5d72db-04fc-4fbc-9db7-959c7458ad90","Type":"ContainerStarted","Data":"8cb86e184ec462c833c66a60e9cdd99bd32dd3d5af84f57747d3a54225ff025b"}
Sep 05 06:26:58.562929 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:58.562907    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-76rz9" event={"ID":"24c5e798-bf33-4f1d-8296-6c4bcff878d6","Type":"ContainerStarted","Data":"826fca712a935cc7968c1097a92298f19baa52dddc53b2203dcea2569313ec19"}
Sep 05 06:26:59.418976 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:59.418811    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:26:59.491506 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:26:59.491416    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:00.360158 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:00.360115    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:00.460338 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:00.460287    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:01.572706 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:01.572672    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-76rz9" event={"ID":"24c5e798-bf33-4f1d-8296-6c4bcff878d6","Type":"ContainerStarted","Data":"89e38b1a7c1ce0da18c63a645564e170de1d688c705862ae22eb03f029a76652"}
Sep 05 06:27:01.573195 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:01.572737    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:27:01.589876 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:01.589652    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-76rz9" podStartSLOduration=66.660419912 podStartE2EDuration="1m9.589630499s" totalImagesPullingTime="2.929210587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:01.589011945 +0000 UTC m=+69.957059155" watchObservedRunningTime="2026-09-05 06:27:01.589630499 +0000 UTC m=+69.957677710"
Sep 05 06:27:03.275533 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:03.275493    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:03.578834 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:03.578748    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cjd64" event={"ID":"ac5d72db-04fc-4fbc-9db7-959c7458ad90","Type":"ContainerStarted","Data":"afbbe8539f5121319d2cccb6d5a7bc4d55304836a3614b8b2ed733e8450d176f"}
Sep 05 06:27:03.596156 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:03.596101    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cjd64" podStartSLOduration=66.205595328 podStartE2EDuration="1m10.596084507s" totalImagesPullingTime="4.390489179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:03.595531013 +0000 UTC m=+71.963578235" watchObservedRunningTime="2026-09-05 06:27:03.596084507 +0000 UTC m=+71.964131712"
Sep 05 06:27:06.350004 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:06.349963    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:16.616914 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:16.616779    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:16.678676 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:16.678628    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:24.327243 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:24.327203    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:24.396263 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:24.396217    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:24.531613 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:24.531578    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:28.264645 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.264611    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-c9dt2"]
Sep 05 06:27:28.269333 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.269305    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-c9dt2"
Sep 05 06:27:28.271870 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.271835    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qzrg6\""
Sep 05 06:27:28.276913 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.276890    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-c9dt2"]
Sep 05 06:27:28.360961 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.360917    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"]
Sep 05 06:27:28.363833 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.363809    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-4n587"]
Sep 05 06:27:28.363992 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.363931    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:28.366360 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.366340    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 05 06:27:28.366498 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.366444    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.367734 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.367712    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 05 06:27:28.367860 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.367746    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 05 06:27:28.367860 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.367806    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nkrcj\""
Sep 05 06:27:28.367981 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.367941    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 05 06:27:28.370386 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.370299    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 05 06:27:28.370386 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.370347    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 05 06:27:28.370515 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.370396    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 05 06:27:28.370614 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.370596    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 05 06:27:28.370796 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.370613    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bnzzz\""
Sep 05 06:27:28.376180 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.376159    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5948\" (UniqueName: \"kubernetes.io/projected/c5321ffe-8938-4f67-aa85-4bdb0d8221ae-kube-api-access-f5948\") pod \"network-check-source-685456655f-c9dt2\" (UID: \"c5321ffe-8938-4f67-aa85-4bdb0d8221ae\") " pod="openshift-network-diagnostics/network-check-source-685456655f-c9dt2"
Sep 05 06:27:28.376668 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.376648    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 05 06:27:28.385078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.385057    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"]
Sep 05 06:27:28.385785 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.385763    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-4n587"]
Sep 05 06:27:28.426057 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.426020    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:28.477277 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.477244    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:28.477366 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.477294    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa4da948-54e4-40a2-8c68-e6a250c5f2be-serving-cert\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.477404 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.477366    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa4da948-54e4-40a2-8c68-e6a250c5f2be-config\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.477438 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.477418    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmh2x\" (UniqueName: \"kubernetes.io/projected/d2cba025-8884-4854-9836-a04f737e6ec1-kube-api-access-pmh2x\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:28.477471 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.477446    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdzw6\" (UniqueName: \"kubernetes.io/projected/aa4da948-54e4-40a2-8c68-e6a250c5f2be-kube-api-access-mdzw6\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.477507 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.477488    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d2cba025-8884-4854-9836-a04f737e6ec1-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:28.477539 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.477511    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa4da948-54e4-40a2-8c68-e6a250c5f2be-trusted-ca\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.477539 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.477529    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5948\" (UniqueName: \"kubernetes.io/projected/c5321ffe-8938-4f67-aa85-4bdb0d8221ae-kube-api-access-f5948\") pod \"network-check-source-685456655f-c9dt2\" (UID: \"c5321ffe-8938-4f67-aa85-4bdb0d8221ae\") " pod="openshift-network-diagnostics/network-check-source-685456655f-c9dt2"
Sep 05 06:27:28.488674 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.488643    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5948\" (UniqueName: \"kubernetes.io/projected/c5321ffe-8938-4f67-aa85-4bdb0d8221ae-kube-api-access-f5948\") pod \"network-check-source-685456655f-c9dt2\" (UID: \"c5321ffe-8938-4f67-aa85-4bdb0d8221ae\") " pod="openshift-network-diagnostics/network-check-source-685456655f-c9dt2"
Sep 05 06:27:28.489058 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.489034    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:28.577445 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.577368    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-c9dt2"
Sep 05 06:27:28.578637 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.578619    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d2cba025-8884-4854-9836-a04f737e6ec1-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:28.578690 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.578649    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa4da948-54e4-40a2-8c68-e6a250c5f2be-trusted-ca\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.578724 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.578701    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:28.578760 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.578726    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa4da948-54e4-40a2-8c68-e6a250c5f2be-serving-cert\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.578792 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.578760    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa4da948-54e4-40a2-8c68-e6a250c5f2be-config\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.578792 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.578780    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmh2x\" (UniqueName: \"kubernetes.io/projected/d2cba025-8884-4854-9836-a04f737e6ec1-kube-api-access-pmh2x\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:28.578895 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.578803    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdzw6\" (UniqueName: \"kubernetes.io/projected/aa4da948-54e4-40a2-8c68-e6a250c5f2be-kube-api-access-mdzw6\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.578895 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:28.578881    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:28.578994 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:28.578960    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls podName:d2cba025-8884-4854-9836-a04f737e6ec1 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:29.078938467 +0000 UTC m=+97.446985668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-d97bv" (UID: "d2cba025-8884-4854-9836-a04f737e6ec1") : secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:28.579373 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.579350    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d2cba025-8884-4854-9836-a04f737e6ec1-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:28.579562 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.579537    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa4da948-54e4-40a2-8c68-e6a250c5f2be-config\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.579638 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.579623    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa4da948-54e4-40a2-8c68-e6a250c5f2be-trusted-ca\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.581140 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.581119    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa4da948-54e4-40a2-8c68-e6a250c5f2be-serving-cert\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.588444 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.588424    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdzw6\" (UniqueName: \"kubernetes.io/projected/aa4da948-54e4-40a2-8c68-e6a250c5f2be-kube-api-access-mdzw6\") pod \"console-operator-85b454874b-4n587\" (UID: \"aa4da948-54e4-40a2-8c68-e6a250c5f2be\") " pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.588551 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.588450    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmh2x\" (UniqueName: \"kubernetes.io/projected/d2cba025-8884-4854-9836-a04f737e6ec1-kube-api-access-pmh2x\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:28.679515 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.679466    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:28.847033 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.846954    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-c9dt2"]
Sep 05 06:27:28.849744 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:27:28.849708    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5321ffe_8938_4f67_aa85_4bdb0d8221ae.slice/crio-ef6a0f841c2451b71e6987f0b1a7274ad5db85b2d6fc1f69ed17de1c772b2745 WatchSource:0}: Error finding container ef6a0f841c2451b71e6987f0b1a7274ad5db85b2d6fc1f69ed17de1c772b2745: Status 404 returned error can't find the container with id ef6a0f841c2451b71e6987f0b1a7274ad5db85b2d6fc1f69ed17de1c772b2745
Sep 05 06:27:28.949649 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:28.949568    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-4n587"]
Sep 05 06:27:28.953189 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:27:28.953153    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa4da948_54e4_40a2_8c68_e6a250c5f2be.slice/crio-618e0f8a2730fbadbf49673afb45e278aec43e56c10ee83c9bdb9bcc1561b1d7 WatchSource:0}: Error finding container 618e0f8a2730fbadbf49673afb45e278aec43e56c10ee83c9bdb9bcc1561b1d7: Status 404 returned error can't find the container with id 618e0f8a2730fbadbf49673afb45e278aec43e56c10ee83c9bdb9bcc1561b1d7
Sep 05 06:27:29.082074 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:29.082033    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:29.082238 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:29.082190    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:29.082274 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:29.082267    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls podName:d2cba025-8884-4854-9836-a04f737e6ec1 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:30.082250284 +0000 UTC m=+98.450297492 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-d97bv" (UID: "d2cba025-8884-4854-9836-a04f737e6ec1") : secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:29.645646 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:29.645606    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-c9dt2" event={"ID":"c5321ffe-8938-4f67-aa85-4bdb0d8221ae","Type":"ContainerStarted","Data":"7dcb18a490b5958085f4086f7d100bf1227cdc7fbf6421e22e9a909c5d72a71a"}
Sep 05 06:27:29.646153 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:29.645666    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-c9dt2" event={"ID":"c5321ffe-8938-4f67-aa85-4bdb0d8221ae","Type":"ContainerStarted","Data":"ef6a0f841c2451b71e6987f0b1a7274ad5db85b2d6fc1f69ed17de1c772b2745"}
Sep 05 06:27:29.647024 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:29.646991    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-4n587" event={"ID":"aa4da948-54e4-40a2-8c68-e6a250c5f2be","Type":"ContainerStarted","Data":"618e0f8a2730fbadbf49673afb45e278aec43e56c10ee83c9bdb9bcc1561b1d7"}
Sep 05 06:27:29.664240 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:29.664191    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-c9dt2" podStartSLOduration=1.664173957 podStartE2EDuration="1.664173957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:29.662840003 +0000 UTC m=+98.030887211" watchObservedRunningTime="2026-09-05 06:27:29.664173957 +0000 UTC m=+98.032221165"
Sep 05 06:27:29.888891 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:29.888840    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:27:29.889068 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:29.888910    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls\") pod \"image-registry-6bc8d97d45-s96zd\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") " pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:27:29.889068 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:29.888998    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 05 06:27:29.889068 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:29.889008    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc8d97d45-s96zd: secret "image-registry-tls" not found
Sep 05 06:27:29.889068 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:29.889007    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:27:29.889068 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:29.889055    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls podName:dbf8d038-21ba-42eb-aac4-7b16c7eae743 nodeName:}" failed. No retries permitted until 2026-09-05 06:28:33.889043005 +0000 UTC m=+162.257090192 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls") pod "image-registry-6bc8d97d45-s96zd" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743") : secret "image-registry-tls" not found
Sep 05 06:27:29.889068 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:29.889069    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls podName:ae33df4f-832d-4ee1-83b7-798f60f540d8 nodeName:}" failed. No retries permitted until 2026-09-05 06:28:33.889062315 +0000 UTC m=+162.257109502 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls") pod "dns-default-xpm9h" (UID: "ae33df4f-832d-4ee1-83b7-798f60f540d8") : secret "dns-default-metrics-tls" not found
Sep 05 06:27:30.090616 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:30.090575    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:30.090787 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:30.090730    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:30.090875 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:30.090795    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls podName:d2cba025-8884-4854-9836-a04f737e6ec1 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:32.090781144 +0000 UTC m=+100.458828335 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-d97bv" (UID: "d2cba025-8884-4854-9836-a04f737e6ec1") : secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:30.451701 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:30.451556    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:31.654144 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:31.654118    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/0.log"
Sep 05 06:27:31.654542 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:31.654154    2573 generic.go:356] "Generic (PLEG): container finished" podID="aa4da948-54e4-40a2-8c68-e6a250c5f2be" containerID="b5d17a62e88b5710e2f5539c2de8740209df77b904dc82185726b7612713694d" exitCode=255
Sep 05 06:27:31.654542 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:31.654211    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-4n587" event={"ID":"aa4da948-54e4-40a2-8c68-e6a250c5f2be","Type":"ContainerDied","Data":"b5d17a62e88b5710e2f5539c2de8740209df77b904dc82185726b7612713694d"}
Sep 05 06:27:31.654542 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:31.654419    2573 scope.go:117] "RemoveContainer" containerID="b5d17a62e88b5710e2f5539c2de8740209df77b904dc82185726b7612713694d"
Sep 05 06:27:31.977341 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:31.977228    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:32.077949 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.077906    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:32.108895 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.108842    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:32.109021 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:32.108987    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:32.109081 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:32.109050    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls podName:d2cba025-8884-4854-9836-a04f737e6ec1 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:36.109034273 +0000 UTC m=+104.477081461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-d97bv" (UID: "d2cba025-8884-4854-9836-a04f737e6ec1") : secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:32.463601 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.463566    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:32.556684 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.556642    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-npbp9"]
Sep 05 06:27:32.556684 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.556684    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:32.559653 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.559635    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.561925 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.561903    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 05 06:27:32.562112 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.562094    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 05 06:27:32.562202 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.562151    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 05 06:27:32.562927 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.562913    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rb49r\""
Sep 05 06:27:32.562989 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.562925    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 05 06:27:32.573954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.573930    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-npbp9"]
Sep 05 06:27:32.578056 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.578032    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-76rz9"
Sep 05 06:27:32.662659 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.662629    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/1.log"
Sep 05 06:27:32.663134 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.663117    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/0.log"
Sep 05 06:27:32.663211 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.663155    2573 generic.go:356] "Generic (PLEG): container finished" podID="aa4da948-54e4-40a2-8c68-e6a250c5f2be" containerID="934e91ad4a32e33e7a867732933d714419a0452d76e83b8a281ef9f472276c03" exitCode=255
Sep 05 06:27:32.663263 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.663209    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-4n587" event={"ID":"aa4da948-54e4-40a2-8c68-e6a250c5f2be","Type":"ContainerDied","Data":"934e91ad4a32e33e7a867732933d714419a0452d76e83b8a281ef9f472276c03"}
Sep 05 06:27:32.663315 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.663281    2573 scope.go:117] "RemoveContainer" containerID="b5d17a62e88b5710e2f5539c2de8740209df77b904dc82185726b7612713694d"
Sep 05 06:27:32.663519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.663498    2573 scope.go:117] "RemoveContainer" containerID="934e91ad4a32e33e7a867732933d714419a0452d76e83b8a281ef9f472276c03"
Sep 05 06:27:32.663697 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:32.663674    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-4n587_openshift-console-operator(aa4da948-54e4-40a2-8c68-e6a250c5f2be)\"" pod="openshift-console-operator/console-operator-85b454874b-4n587" podUID="aa4da948-54e4-40a2-8c68-e6a250c5f2be"
Sep 05 06:27:32.682124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.682088    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:32.714156 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.714067    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9npg\" (UniqueName: \"kubernetes.io/projected/6df16279-9467-4374-b506-15aeed9823c0-kube-api-access-w9npg\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.714156 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.714119    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6df16279-9467-4374-b506-15aeed9823c0-crio-socket\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.714156 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.714147    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6df16279-9467-4374-b506-15aeed9823c0-data-volume\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.714480 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.714170    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.714480 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.714192    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.815522 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.815490    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9npg\" (UniqueName: \"kubernetes.io/projected/6df16279-9467-4374-b506-15aeed9823c0-kube-api-access-w9npg\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.815522 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.815533    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6df16279-9467-4374-b506-15aeed9823c0-crio-socket\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.815777 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.815555    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6df16279-9467-4374-b506-15aeed9823c0-data-volume\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.815777 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.815647    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6df16279-9467-4374-b506-15aeed9823c0-crio-socket\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.815914 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.815786    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.815914 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.815838    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.815914 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.815907    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6df16279-9467-4374-b506-15aeed9823c0-data-volume\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.816077 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:32.815944    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 06:27:32.816077 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:32.815988    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:32.816077 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:32.816005    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm podName:6df16279-9467-4374-b506-15aeed9823c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:33.315989278 +0000 UTC m=+101.684036465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-npbp9" (UID: "6df16279-9467-4374-b506-15aeed9823c0") : configmap "kube-rbac-proxy" not found
Sep 05 06:27:32.816077 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:32.816034    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls podName:6df16279-9467-4374-b506-15aeed9823c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:33.316022086 +0000 UTC m=+101.684069274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-npbp9" (UID: "6df16279-9467-4374-b506-15aeed9823c0") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:32.824458 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.824438    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9npg\" (UniqueName: \"kubernetes.io/projected/6df16279-9467-4374-b506-15aeed9823c0-kube-api-access-w9npg\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:32.895283 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.895227    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:32.983869 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:32.983778    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:33.065311 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:33.065273    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:33.162293 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:33.162258    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:33.262100 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:33.262064    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:33.321179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:33.321146    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:33.321336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:33.321194    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:33.321336 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:33.321288    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 06:27:33.321405 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:33.321362    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm podName:6df16279-9467-4374-b506-15aeed9823c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:34.321345033 +0000 UTC m=+102.689392225 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-npbp9" (UID: "6df16279-9467-4374-b506-15aeed9823c0") : configmap "kube-rbac-proxy" not found
Sep 05 06:27:33.321405 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:33.321369    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:33.321471 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:33.321423    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls podName:6df16279-9467-4374-b506-15aeed9823c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:34.321407084 +0000 UTC m=+102.689454273 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-npbp9" (UID: "6df16279-9467-4374-b506-15aeed9823c0") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:33.666972 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:33.666902    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/1.log"
Sep 05 06:27:33.667347 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:33.667196    2573 scope.go:117] "RemoveContainer" containerID="934e91ad4a32e33e7a867732933d714419a0452d76e83b8a281ef9f472276c03"
Sep 05 06:27:33.667347 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:33.667336    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-4n587_openshift-console-operator(aa4da948-54e4-40a2-8c68-e6a250c5f2be)\"" pod="openshift-console-operator/console-operator-85b454874b-4n587" podUID="aa4da948-54e4-40a2-8c68-e6a250c5f2be"
Sep 05 06:27:34.330497 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.330437    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:34.330750 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.330589    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:34.330750 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:34.330599    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:34.330750 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:34.330659    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 06:27:34.330750 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:34.330662    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls podName:6df16279-9467-4374-b506-15aeed9823c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:36.330644276 +0000 UTC m=+104.698691480 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-npbp9" (UID: "6df16279-9467-4374-b506-15aeed9823c0") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:34.330750 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:34.330701    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm podName:6df16279-9467-4374-b506-15aeed9823c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:36.330690654 +0000 UTC m=+104.698737842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-npbp9" (UID: "6df16279-9467-4374-b506-15aeed9823c0") : configmap "kube-rbac-proxy" not found
Sep 05 06:27:34.529832 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.529802    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nq8t2_e7095477-5141-481a-8b27-64e4c36845cb/dns-node-resolver/0.log"
Sep 05 06:27:34.780765 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.780735    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-bqjxl"]
Sep 05 06:27:34.813472 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.813437    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-bqjxl"]
Sep 05 06:27:34.813621 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.813536    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-bqjxl"
Sep 05 06:27:34.816959 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.816936    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 05 06:27:34.817086 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.816959    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 05 06:27:34.817658 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.817642    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-fdjmc\""
Sep 05 06:27:34.817721 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.817678    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 05 06:27:34.817765 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.817738    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 05 06:27:34.827496 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.827472    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:34.903573 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.903532    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:34.935939 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.935903    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d43951d7-1528-4b93-8dff-55e14fb1e2ad-signing-key\") pod \"service-ca-75d99c484-bqjxl\" (UID: \"d43951d7-1528-4b93-8dff-55e14fb1e2ad\") " pod="openshift-service-ca/service-ca-75d99c484-bqjxl"
Sep 05 06:27:34.936120 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.935946    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zrq4\" (UniqueName: \"kubernetes.io/projected/d43951d7-1528-4b93-8dff-55e14fb1e2ad-kube-api-access-9zrq4\") pod \"service-ca-75d99c484-bqjxl\" (UID: \"d43951d7-1528-4b93-8dff-55e14fb1e2ad\") " pod="openshift-service-ca/service-ca-75d99c484-bqjxl"
Sep 05 06:27:34.936120 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:34.936021    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d43951d7-1528-4b93-8dff-55e14fb1e2ad-signing-cabundle\") pod \"service-ca-75d99c484-bqjxl\" (UID: \"d43951d7-1528-4b93-8dff-55e14fb1e2ad\") " pod="openshift-service-ca/service-ca-75d99c484-bqjxl"
Sep 05 06:27:35.037121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:35.037040    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d43951d7-1528-4b93-8dff-55e14fb1e2ad-signing-key\") pod \"service-ca-75d99c484-bqjxl\" (UID: \"d43951d7-1528-4b93-8dff-55e14fb1e2ad\") " pod="openshift-service-ca/service-ca-75d99c484-bqjxl"
Sep 05 06:27:35.037121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:35.037078    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zrq4\" (UniqueName: \"kubernetes.io/projected/d43951d7-1528-4b93-8dff-55e14fb1e2ad-kube-api-access-9zrq4\") pod \"service-ca-75d99c484-bqjxl\" (UID: \"d43951d7-1528-4b93-8dff-55e14fb1e2ad\") " pod="openshift-service-ca/service-ca-75d99c484-bqjxl"
Sep 05 06:27:35.037337 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:35.037189    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d43951d7-1528-4b93-8dff-55e14fb1e2ad-signing-cabundle\") pod \"service-ca-75d99c484-bqjxl\" (UID: \"d43951d7-1528-4b93-8dff-55e14fb1e2ad\") " pod="openshift-service-ca/service-ca-75d99c484-bqjxl"
Sep 05 06:27:35.037898 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:35.037880    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d43951d7-1528-4b93-8dff-55e14fb1e2ad-signing-cabundle\") pod \"service-ca-75d99c484-bqjxl\" (UID: \"d43951d7-1528-4b93-8dff-55e14fb1e2ad\") " pod="openshift-service-ca/service-ca-75d99c484-bqjxl"
Sep 05 06:27:35.039506 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:35.039483    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d43951d7-1528-4b93-8dff-55e14fb1e2ad-signing-key\") pod \"service-ca-75d99c484-bqjxl\" (UID: \"d43951d7-1528-4b93-8dff-55e14fb1e2ad\") " pod="openshift-service-ca/service-ca-75d99c484-bqjxl"
Sep 05 06:27:35.047157 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:35.047137    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zrq4\" (UniqueName: \"kubernetes.io/projected/d43951d7-1528-4b93-8dff-55e14fb1e2ad-kube-api-access-9zrq4\") pod \"service-ca-75d99c484-bqjxl\" (UID: \"d43951d7-1528-4b93-8dff-55e14fb1e2ad\") " pod="openshift-service-ca/service-ca-75d99c484-bqjxl"
Sep 05 06:27:35.121819 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:35.121777    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-bqjxl"
Sep 05 06:27:35.329364 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:35.329287    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7pdxl_4bad5cfd-a364-42c1-90dd-e2f7ddc39e64/node-ca/0.log"
Sep 05 06:27:35.422055 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:35.422032    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-bqjxl"]
Sep 05 06:27:35.424107 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:27:35.424074    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43951d7_1528_4b93_8dff_55e14fb1e2ad.slice/crio-3556a64f7ff297bc07c9219216e123ee2ea0b00757ad03ecce335a282065aab5 WatchSource:0}: Error finding container 3556a64f7ff297bc07c9219216e123ee2ea0b00757ad03ecce335a282065aab5: Status 404 returned error can't find the container with id 3556a64f7ff297bc07c9219216e123ee2ea0b00757ad03ecce335a282065aab5
Sep 05 06:27:35.674037 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:35.673955    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-bqjxl" event={"ID":"d43951d7-1528-4b93-8dff-55e14fb1e2ad","Type":"ContainerStarted","Data":"3556a64f7ff297bc07c9219216e123ee2ea0b00757ad03ecce335a282065aab5"}
Sep 05 06:27:35.770248 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:35.770218    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:36.147864 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:36.147815    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:36.148288 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:36.147983    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:36.148288 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:36.148051    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls podName:d2cba025-8884-4854-9836-a04f737e6ec1 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:44.148033796 +0000 UTC m=+112.516080989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-d97bv" (UID: "d2cba025-8884-4854-9836-a04f737e6ec1") : secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:36.349697 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:36.349660    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:36.349697 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:36.349700    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:36.349947 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:36.349813    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 06:27:36.349947 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:36.349842    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:36.349947 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:36.349899    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm podName:6df16279-9467-4374-b506-15aeed9823c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:40.349879734 +0000 UTC m=+108.717926942 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-npbp9" (UID: "6df16279-9467-4374-b506-15aeed9823c0") : configmap "kube-rbac-proxy" not found
Sep 05 06:27:36.349947 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:36.349919    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls podName:6df16279-9467-4374-b506-15aeed9823c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:40.349909604 +0000 UTC m=+108.717956799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-npbp9" (UID: "6df16279-9467-4374-b506-15aeed9823c0") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:36.529166 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:36.529123    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-llbmp_5d2c676f-9881-4180-9aa4-2cc3757adfa0/serve-healthcheck-canary/0.log"
Sep 05 06:27:37.681086 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:37.681051    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-bqjxl" event={"ID":"d43951d7-1528-4b93-8dff-55e14fb1e2ad","Type":"ContainerStarted","Data":"87ec0b5e159f7d76c560c1b1b33d253aa422f11530daf8a418ce872232367eb2"}
Sep 05 06:27:37.698598 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:37.698536    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-bqjxl" podStartSLOduration=1.623896145 podStartE2EDuration="3.698515318s" totalImagesPullingTime="2.074619173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:37.697629852 +0000 UTC m=+106.065677062" watchObservedRunningTime="2026-09-05 06:27:37.698515318 +0000 UTC m=+106.066562532"
Sep 05 06:27:38.644258 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:38.644222    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:38.680032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:38.679988    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:38.680238 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:38.680120    2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:38.680581 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:38.680565    2573 scope.go:117] "RemoveContainer" containerID="934e91ad4a32e33e7a867732933d714419a0452d76e83b8a281ef9f472276c03"
Sep 05 06:27:38.680770 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:38.680750    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-4n587_openshift-console-operator(aa4da948-54e4-40a2-8c68-e6a250c5f2be)\"" pod="openshift-console-operator/console-operator-85b454874b-4n587" podUID="aa4da948-54e4-40a2-8c68-e6a250c5f2be"
Sep 05 06:27:38.759507 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:38.759469    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:38.857504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:38.857468    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:38.958590 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:38.958499    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:39.102632 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:39.102591    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:39.197529 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:39.197493    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:39.298004 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:39.297957    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:39.403903 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:39.403873    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:40.139290 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:40.139251    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:40.384217 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:40.384173    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:40.384217 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:40.384215    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:40.384416 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:40.384308    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 06:27:40.384416 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:40.384343    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:40.384416 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:40.384372    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm podName:6df16279-9467-4374-b506-15aeed9823c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:48.384357361 +0000 UTC m=+116.752404548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-npbp9" (UID: "6df16279-9467-4374-b506-15aeed9823c0") : configmap "kube-rbac-proxy" not found
Sep 05 06:27:40.384416 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:40.384387    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls podName:6df16279-9467-4374-b506-15aeed9823c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:48.384381242 +0000 UTC m=+116.752428429 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-npbp9" (UID: "6df16279-9467-4374-b506-15aeed9823c0") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:40.537967 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:40.537934    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:40.951076 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:40.950984    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:41.138316 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:41.138283    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:41.733232 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:41.733191    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:42.143689 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:42.143645    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:42.333723 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:42.333680    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:42.403204 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:42.403123    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:42.676128 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:42.676040    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:42.777314 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:42.777275    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:42.938539 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:42.938438    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:43.136326 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:43.136293    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:43.201998 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:43.201909    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:43.737261 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:43.737200    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:44.137744 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:44.137706    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:44.216316 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:44.216277    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:27:44.216491 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:44.216447    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:44.216586 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:44.216525    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls podName:d2cba025-8884-4854-9836-a04f737e6ec1 nodeName:}" failed. No retries permitted until 2026-09-05 06:28:00.216504913 +0000 UTC m=+128.584552114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-d97bv" (UID: "d2cba025-8884-4854-9836-a04f737e6ec1") : secret "cluster-monitoring-operator-tls" not found
Sep 05 06:27:44.538641 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:44.538607    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:44.932669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:44.932586    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:45.033197 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:45.033164    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:45.165911 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:45.165876    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:45.340745 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:45.340711    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:45.725997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:45.725901    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:46.150279 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:46.150242    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:46.677695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:46.677660    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:46.738840 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:46.738809    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:46.933662 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:46.933585    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:47.139290 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:47.139248    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:47.547749 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:47.547716    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:48.142753 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:48.142720    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:48.451697 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:48.451601    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:48.451697 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:48.451650    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:48.451916 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:48.451764    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:48.451916 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:48.451824    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls podName:6df16279-9467-4374-b506-15aeed9823c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:28:04.451808043 +0000 UTC m=+132.819855252 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-npbp9" (UID: "6df16279-9467-4374-b506-15aeed9823c0") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:48.452187 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:48.452163    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6df16279-9467-4374-b506-15aeed9823c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:27:48.546060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:48.543817    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:48.937892 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:48.937840    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:49.135768 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:49.135725    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:49.149990 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:49.149966    2573 scope.go:117] "RemoveContainer" containerID="934e91ad4a32e33e7a867732933d714419a0452d76e83b8a281ef9f472276c03"
Sep 05 06:27:49.331679 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:49.331647    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:49.720361 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:49.720332    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:27:49.720750 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:49.720735    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/1.log"
Sep 05 06:27:49.720796 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:49.720768    2573 generic.go:356] "Generic (PLEG): container finished" podID="aa4da948-54e4-40a2-8c68-e6a250c5f2be" containerID="c7548b8db70e21437077b2a427ae9aa44b1de6a40ae10558908079198ec9a576" exitCode=255
Sep 05 06:27:49.720842 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:49.720827    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-4n587" event={"ID":"aa4da948-54e4-40a2-8c68-e6a250c5f2be","Type":"ContainerDied","Data":"c7548b8db70e21437077b2a427ae9aa44b1de6a40ae10558908079198ec9a576"}
Sep 05 06:27:49.720913 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:49.720895    2573 scope.go:117] "RemoveContainer" containerID="934e91ad4a32e33e7a867732933d714419a0452d76e83b8a281ef9f472276c03"
Sep 05 06:27:49.721191 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:49.721169    2573 scope.go:117] "RemoveContainer" containerID="c7548b8db70e21437077b2a427ae9aa44b1de6a40ae10558908079198ec9a576"
Sep 05 06:27:49.721384 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:49.721359    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-4n587_openshift-console-operator(aa4da948-54e4-40a2-8c68-e6a250c5f2be)\"" pod="openshift-console-operator/console-operator-85b454874b-4n587" podUID="aa4da948-54e4-40a2-8c68-e6a250c5f2be"
Sep 05 06:27:50.724642 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:50.724610    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:27:56.914772 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:56.914738    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5"]
Sep 05 06:27:56.918942 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:56.918923    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5"
Sep 05 06:27:56.921506 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:56.921486    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-sk2ks\""
Sep 05 06:27:56.921735 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:56.921710    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 05 06:27:56.922902 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:56.922887    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 05 06:27:56.923002 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:56.922905    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 05 06:27:56.923002 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:56.922886    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 05 06:27:56.929451 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:56.929431    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5"]
Sep 05 06:27:57.016536 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:57.016501    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qnkf\" (UniqueName: \"kubernetes.io/projected/f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb-kube-api-access-5qnkf\") pod \"managed-serviceaccount-addon-agent-66cf768888-4cvd5\" (UID: \"f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5"
Sep 05 06:27:57.016720 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:57.016555    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66cf768888-4cvd5\" (UID: \"f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5"
Sep 05 06:27:57.053613 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:57.053578    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:57.117962 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:57.117923    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qnkf\" (UniqueName: \"kubernetes.io/projected/f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb-kube-api-access-5qnkf\") pod \"managed-serviceaccount-addon-agent-66cf768888-4cvd5\" (UID: \"f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5"
Sep 05 06:27:57.118155 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:57.117995    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66cf768888-4cvd5\" (UID: \"f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5"
Sep 05 06:27:57.120475 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:57.120456    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66cf768888-4cvd5\" (UID: \"f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5"
Sep 05 06:27:57.127406 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:57.127381    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qnkf\" (UniqueName: \"kubernetes.io/projected/f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb-kube-api-access-5qnkf\") pod \"managed-serviceaccount-addon-agent-66cf768888-4cvd5\" (UID: \"f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5"
Sep 05 06:27:57.147403 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:57.147371    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:27:57.238357 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:57.238267    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5"
Sep 05 06:27:57.507923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:57.507892    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5"]
Sep 05 06:27:57.511403 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:27:57.511369    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5305efb_b4a6_45e6_bfa7_5d4968dcc9eb.slice/crio-ac7c614b3c4032dbe595e5b336064569369074e96810c23ce8e1a76571d45c0a WatchSource:0}: Error finding container ac7c614b3c4032dbe595e5b336064569369074e96810c23ce8e1a76571d45c0a: Status 404 returned error can't find the container with id ac7c614b3c4032dbe595e5b336064569369074e96810c23ce8e1a76571d45c0a
Sep 05 06:27:57.747394 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:57.746807    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5" event={"ID":"f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb","Type":"ContainerStarted","Data":"ac7c614b3c4032dbe595e5b336064569369074e96810c23ce8e1a76571d45c0a"}
Sep 05 06:27:58.680604 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:58.680567    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:58.681089 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:58.680641    2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:27:58.681089 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:27:58.681028    2573 scope.go:117] "RemoveContainer" containerID="c7548b8db70e21437077b2a427ae9aa44b1de6a40ae10558908079198ec9a576"
Sep 05 06:27:58.681261 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:27:58.681233    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-4n587_openshift-console-operator(aa4da948-54e4-40a2-8c68-e6a250c5f2be)\"" pod="openshift-console-operator/console-operator-85b454874b-4n587" podUID="aa4da948-54e4-40a2-8c68-e6a250c5f2be"
Sep 05 06:28:00.243276 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:00.243240    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:28:00.245610 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:00.245587    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2cba025-8884-4854-9836-a04f737e6ec1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-d97bv\" (UID: \"d2cba025-8884-4854-9836-a04f737e6ec1\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:28:00.477169 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:00.477135    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nkrcj\""
Sep 05 06:28:00.485356 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:00.485338    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"
Sep 05 06:28:00.757344 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:00.757246    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv"]
Sep 05 06:28:00.758645 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:00.758584    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5" event={"ID":"f5305efb-b4a6-45e6-bfa7-5d4968dcc9eb","Type":"ContainerStarted","Data":"b8a1a77fe8234cdd0e3835c14ede89ee489d364401614ff3a5c72f5fb2575eb1"}
Sep 05 06:28:00.760471 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:28:00.760449    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2cba025_8884_4854_9836_a04f737e6ec1.slice/crio-46aa79adb952abb261f7ec084119ea404de7b3f15bc6c54738297824f561f0bd WatchSource:0}: Error finding container 46aa79adb952abb261f7ec084119ea404de7b3f15bc6c54738297824f561f0bd: Status 404 returned error can't find the container with id 46aa79adb952abb261f7ec084119ea404de7b3f15bc6c54738297824f561f0bd
Sep 05 06:28:00.776571 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:00.776507    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66cf768888-4cvd5" podStartSLOduration=2.396068264 podStartE2EDuration="4.776491246s" totalImagesPullingTime="2.380422982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:00.775868039 +0000 UTC m=+129.143915238" watchObservedRunningTime="2026-09-05 06:28:00.776491246 +0000 UTC m=+129.144538446"
Sep 05 06:28:01.762790 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:01.762749    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv" event={"ID":"d2cba025-8884-4854-9836-a04f737e6ec1","Type":"ContainerStarted","Data":"46aa79adb952abb261f7ec084119ea404de7b3f15bc6c54738297824f561f0bd"}
Sep 05 06:28:01.812306 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:01.812275    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:01.912925 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:01.912891    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:01.959158 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:01.959119    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:28:01.961610 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:01.961584    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e75f6f78-89ec-47e6-a11c-910a4362182d-metrics-certs\") pod \"network-metrics-daemon-75v4r\" (UID: \"e75f6f78-89ec-47e6-a11c-910a4362182d\") " pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:28:02.165431 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.165343    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4m5kh\""
Sep 05 06:28:02.172793 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.172766    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-75v4r"
Sep 05 06:28:02.536306 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.536263    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-75v4r"]
Sep 05 06:28:02.540423 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:28:02.540392    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode75f6f78_89ec_47e6_a11c_910a4362182d.slice/crio-15d81b0d1749a3f7ca5df51e3818ff23fedb206aabeeb24df01d7e5669962368 WatchSource:0}: Error finding container 15d81b0d1749a3f7ca5df51e3818ff23fedb206aabeeb24df01d7e5669962368: Status 404 returned error can't find the container with id 15d81b0d1749a3f7ca5df51e3818ff23fedb206aabeeb24df01d7e5669962368
Sep 05 06:28:02.660165 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.660133    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:02.728451 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.728401    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:02.768335 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.768301    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv" event={"ID":"d2cba025-8884-4854-9836-a04f737e6ec1","Type":"ContainerStarted","Data":"45eef4d798134d9815c055788dee5bfd6ccd65f494f4eb28a787a45822a2172a"}
Sep 05 06:28:02.771031 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.770991    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-75v4r" event={"ID":"e75f6f78-89ec-47e6-a11c-910a4362182d","Type":"ContainerStarted","Data":"15d81b0d1749a3f7ca5df51e3818ff23fedb206aabeeb24df01d7e5669962368"}
Sep 05 06:28:02.830465 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.830375    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:02.844282 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.844223    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-d97bv" podStartSLOduration=33.284830047 podStartE2EDuration="34.844201525s" totalImagesPullingTime="1.559371478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:02.788610489 +0000 UTC m=+131.156657699" watchObservedRunningTime="2026-09-05 06:28:02.844201525 +0000 UTC m=+131.212248735"
Sep 05 06:28:02.844826 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.844809    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6"]
Sep 05 06:28:02.847941 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.847915    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6"
Sep 05 06:28:02.850496 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.850459    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 05 06:28:02.850653 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.850635    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4qk27\""
Sep 05 06:28:02.856265 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.856243    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6"]
Sep 05 06:28:02.866736 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.866704    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/276610f9-7096-4300-ba3d-35e5088a8b62-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gv7p6\" (UID: \"276610f9-7096-4300-ba3d-35e5088a8b62\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6"
Sep 05 06:28:02.911179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.911140    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:02.967690 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.967653    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/276610f9-7096-4300-ba3d-35e5088a8b62-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gv7p6\" (UID: \"276610f9-7096-4300-ba3d-35e5088a8b62\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6"
Sep 05 06:28:02.970322 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:02.970297    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/276610f9-7096-4300-ba3d-35e5088a8b62-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gv7p6\" (UID: \"276610f9-7096-4300-ba3d-35e5088a8b62\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6"
Sep 05 06:28:03.157503 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:03.157415    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6"
Sep 05 06:28:03.429762 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:03.429690    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6"]
Sep 05 06:28:03.453297 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:28:03.453236    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod276610f9_7096_4300_ba3d_35e5088a8b62.slice/crio-7d30917802cf4fa186c4c716f423a8f6629604b3219badaed614077e9bb661ab WatchSource:0}: Error finding container 7d30917802cf4fa186c4c716f423a8f6629604b3219badaed614077e9bb661ab: Status 404 returned error can't find the container with id 7d30917802cf4fa186c4c716f423a8f6629604b3219badaed614077e9bb661ab
Sep 05 06:28:03.777969 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:03.777932    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-75v4r" event={"ID":"e75f6f78-89ec-47e6-a11c-910a4362182d","Type":"ContainerStarted","Data":"76ad43fffa081518ff48f5e7e07e53add369fa729d7d371e5e46f5cd42c49c20"}
Sep 05 06:28:03.778435 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:03.777995    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-75v4r" event={"ID":"e75f6f78-89ec-47e6-a11c-910a4362182d","Type":"ContainerStarted","Data":"8f3cdf4e9134338f24a9ae007ea4becf7d8b11dbf7981d6c23ef8a71615a9938"}
Sep 05 06:28:03.779088 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:03.779064    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6" event={"ID":"276610f9-7096-4300-ba3d-35e5088a8b62","Type":"ContainerStarted","Data":"7d30917802cf4fa186c4c716f423a8f6629604b3219badaed614077e9bb661ab"}
Sep 05 06:28:03.797684 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:03.797630    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-75v4r" podStartSLOduration=130.88058706 podStartE2EDuration="2m11.797613436s" totalImagesPullingTime="917.026376ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:03.795996185 +0000 UTC m=+132.164043393" watchObservedRunningTime="2026-09-05 06:28:03.797613436 +0000 UTC m=+132.165660645"
Sep 05 06:28:03.927665 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:03.927626    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:04.020166 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:04.020128    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:04.480042 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:04.480012    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:28:04.482251 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:04.482228    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6df16279-9467-4374-b506-15aeed9823c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npbp9\" (UID: \"6df16279-9467-4374-b506-15aeed9823c0\") " pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:28:04.671185 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:04.671155    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rb49r\""
Sep 05 06:28:04.679338 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:04.679314    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-npbp9"
Sep 05 06:28:04.783459 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:04.783420    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6" event={"ID":"276610f9-7096-4300-ba3d-35e5088a8b62","Type":"ContainerStarted","Data":"8fced348582a9c6ad0e69f15da148c0a2d93351dfacd51fb23c2e439f340acdc"}
Sep 05 06:28:04.783924 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:04.783654    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6"
Sep 05 06:28:04.788230 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:04.788206    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6"
Sep 05 06:28:04.837208 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:04.837151    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gv7p6" podStartSLOduration=1.847253966 podStartE2EDuration="2.837134348s" totalImagesPullingTime="989.880382ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:04.812612813 +0000 UTC m=+133.180660023" watchObservedRunningTime="2026-09-05 06:28:04.837134348 +0000 UTC m=+133.205181557"
Sep 05 06:28:04.952136 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:04.951951    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-npbp9"]
Sep 05 06:28:04.954589 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:28:04.954564    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df16279_9467_4374_b506_15aeed9823c0.slice/crio-304f58ff2dd52804e10c01de1f7a4916b2653c2bb674b6da52ce14c04541b418 WatchSource:0}: Error finding container 304f58ff2dd52804e10c01de1f7a4916b2653c2bb674b6da52ce14c04541b418: Status 404 returned error can't find the container with id 304f58ff2dd52804e10c01de1f7a4916b2653c2bb674b6da52ce14c04541b418
Sep 05 06:28:05.790989 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.790954    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-npbp9" event={"ID":"6df16279-9467-4374-b506-15aeed9823c0","Type":"ContainerStarted","Data":"1dcf7ef5c13ac86165d9f4ec16e036b9c95a66660d5ac9b8400650d00631cd2c"}
Sep 05 06:28:05.791387 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.791013    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-npbp9" event={"ID":"6df16279-9467-4374-b506-15aeed9823c0","Type":"ContainerStarted","Data":"b1cff1bd553df933d4ac5e264b2ec2c0e8b3ebb881b4412327d52b4471a82c1e"}
Sep 05 06:28:05.791387 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.791032    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-npbp9" event={"ID":"6df16279-9467-4374-b506-15aeed9823c0","Type":"ContainerStarted","Data":"304f58ff2dd52804e10c01de1f7a4916b2653c2bb674b6da52ce14c04541b418"}
Sep 05 06:28:05.892512 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.892422    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:05.938231 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.938193    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"]
Sep 05 06:28:05.941390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.941373    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:05.944479 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.944458    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-4v2fk\""
Sep 05 06:28:05.944742 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.944720    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 05 06:28:05.945606 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.945588    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 05 06:28:05.945698 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.945606    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 05 06:28:05.954944 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.954922    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"]
Sep 05 06:28:05.992511 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.992475    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:05.992691 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.992551    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:05.992691 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.992581    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:05.992691 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.992603    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q49xs\" (UniqueName: \"kubernetes.io/projected/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-kube-api-access-q49xs\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:05.997428 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:05.997393    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:06.093987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:06.093941    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:06.094174 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:06.094006    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:06.094174 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:06.094032    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q49xs\" (UniqueName: \"kubernetes.io/projected/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-kube-api-access-q49xs\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:06.094174 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:06.094071    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:06.094174 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:28:06.094162    2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 05 06:28:06.094383 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:28:06.094233    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-prometheus-operator-tls podName:5339d188-3c4e-49bc-8ed0-d1f8eda83ff9 nodeName:}" failed. No retries permitted until 2026-09-05 06:28:06.594213259 +0000 UTC m=+134.962260468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-ldfkm" (UID: "5339d188-3c4e-49bc-8ed0-d1f8eda83ff9") : secret "prometheus-operator-tls" not found
Sep 05 06:28:06.094918 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:06.094898    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:06.096741 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:06.096715    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:06.106207 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:06.106183    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q49xs\" (UniqueName: \"kubernetes.io/projected/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-kube-api-access-q49xs\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:06.223874 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:06.223781    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:06.597997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:06.597954    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:06.600599 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:06.600569    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5339d188-3c4e-49bc-8ed0-d1f8eda83ff9-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ldfkm\" (UID: \"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:06.850722 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:06.850645    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"
Sep 05 06:28:07.155463 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:07.155382    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ldfkm"]
Sep 05 06:28:07.158386 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:28:07.158360    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5339d188_3c4e_49bc_8ed0_d1f8eda83ff9.slice/crio-ca579ea753c25c6109f6a7a2a40d4ced9bc2b4231bb664c2738d60ac720465b9 WatchSource:0}: Error finding container ca579ea753c25c6109f6a7a2a40d4ced9bc2b4231bb664c2738d60ac720465b9: Status 404 returned error can't find the container with id ca579ea753c25c6109f6a7a2a40d4ced9bc2b4231bb664c2738d60ac720465b9
Sep 05 06:28:07.801051 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:07.801014    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-npbp9" event={"ID":"6df16279-9467-4374-b506-15aeed9823c0","Type":"ContainerStarted","Data":"b5423793cd050a2afebc80b7585d5e7bc21a2abc9812ecf0c88c9ae12cd6cb50"}
Sep 05 06:28:07.802370 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:07.802336    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm" event={"ID":"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9","Type":"ContainerStarted","Data":"ca579ea753c25c6109f6a7a2a40d4ced9bc2b4231bb664c2738d60ac720465b9"}
Sep 05 06:28:07.823962 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:07.823911    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-npbp9" podStartSLOduration=34.018796477 podStartE2EDuration="35.823893035s" totalImagesPullingTime="1.805096558s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:27:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:07.822199161 +0000 UTC m=+136.190246383" watchObservedRunningTime="2026-09-05 06:28:07.823893035 +0000 UTC m=+136.191940243"
Sep 05 06:28:08.527549 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:08.527467    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:08.808910 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:08.808773    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm" event={"ID":"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9","Type":"ContainerStarted","Data":"a5941d054c64b1bd0c3fd6d9b7dc5ac993fbffbd73aced416563c3edcef6f7b5"}
Sep 05 06:28:08.808910 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:08.808837    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm" event={"ID":"5339d188-3c4e-49bc-8ed0-d1f8eda83ff9","Type":"ContainerStarted","Data":"9cbfd5cf68f485c7a4ac8da1c283a533c74f88de07548413b757d801d90d8530"}
Sep 05 06:28:08.829751 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:08.829703    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-ldfkm" podStartSLOduration=2.730853831 podStartE2EDuration="3.829683486s" totalImagesPullingTime="1.098829655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:08.828827571 +0000 UTC m=+137.196874779" watchObservedRunningTime="2026-09-05 06:28:08.829683486 +0000 UTC m=+137.197730696"
Sep 05 06:28:10.060488 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.060453    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:10.276493 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.276453    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:10.352320 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.352228    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ln2dx"]
Sep 05 06:28:10.359007 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.358984    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.362397 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.362371    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 05 06:28:10.362536 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.362421    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bcsr2\""
Sep 05 06:28:10.362689 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.362674    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 05 06:28:10.363195 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.363180    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 05 06:28:10.393919 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.393878    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:10.431407 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.431363    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-metrics-client-ca\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.431611 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.431422    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-textfile\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.431611 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.431487    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-sys\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.431611 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.431553    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-accelerators-collector-config\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.431611 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.431581    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-wtmp\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.431802 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.431614    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-tls\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.431802 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.431673    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.431802 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.431712    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-root\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.431802 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.431738    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xccsc\" (UniqueName: \"kubernetes.io/projected/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-kube-api-access-xccsc\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.468067 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.468033    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:10.532501 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.532460    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-metrics-client-ca\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.532501 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.532511    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-textfile\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.532778 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.532626    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-sys\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.532778 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.532683    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-accelerators-collector-config\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.532778 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.532703    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-wtmp\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.532778 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.532727    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-tls\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.532778 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.532771    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.533071 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.532806    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-root\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.533071 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.532830    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xccsc\" (UniqueName: \"kubernetes.io/projected/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-kube-api-access-xccsc\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.533071 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.532987    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-textfile\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.533353 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.533246    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-accelerators-collector-config\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.533353 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.533307    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-root\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.533503 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.533401    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-wtmp\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.533503 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.533472    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-sys\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.533584 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.533508    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-metrics-client-ca\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.535334 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.535307    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-tls\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.535427 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.535335    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.541761 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.541736    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xccsc\" (UniqueName: \"kubernetes.io/projected/16ede8b1-46ac-4eb5-811a-bf567cd0dbf9-kube-api-access-xccsc\") pod \"node-exporter-ln2dx\" (UID: \"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9\") " pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.641966 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.641875    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:10.669813 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.669776    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ln2dx"
Sep 05 06:28:10.752927 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.752893    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:10.816642 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.816594    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ln2dx" event={"ID":"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9","Type":"ContainerStarted","Data":"c2f87cfd312c6aa2c76d24586554f75447e89654a471013993c73cceb105fcd4"}
Sep 05 06:28:10.860066 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:10.860036    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:11.124998 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.124963    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:11.150509 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.150466    2573 scope.go:117] "RemoveContainer" containerID="c7548b8db70e21437077b2a427ae9aa44b1de6a40ae10558908079198ec9a576"
Sep 05 06:28:11.303792 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.303761    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:11.393320 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.393150    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:11.400464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.400432    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 06:28:11.408155 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.405171    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.409396 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.409227    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 05 06:28:11.410890 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.410289    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 05 06:28:11.410890 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.410371    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 05 06:28:11.410890 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.410512    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-72zm4\""
Sep 05 06:28:11.410890 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.410656    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 05 06:28:11.410890 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.410828    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 05 06:28:11.410890 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.410840    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 05 06:28:11.419533 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.419461    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 05 06:28:11.419956 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.419686    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 05 06:28:11.419956 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.419771    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 05 06:28:11.430537 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.430397    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 06:28:11.442610 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442576    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.442610 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442610    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zbfm\" (UniqueName: \"kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-kube-api-access-6zbfm\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.442825 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442640    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.442825 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442661    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.442825 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442709    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.442825 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442783    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.442825 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442819    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.443102 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442884    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-web-config\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.443102 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442914    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.443102 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442937    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.443102 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442966    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.443102 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.442992    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-config-out\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.443102 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.443009    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-config-volume\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.544419 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544387    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.544419 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544421    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zbfm\" (UniqueName: \"kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-kube-api-access-6zbfm\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.544669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544446    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.544669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544464    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.544669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544490    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.544669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544541    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.544669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544570    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.544669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544602    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-web-config\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.544669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544630    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.544669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544656    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.545132 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544681    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.545132 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544711    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-config-out\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.545132 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.544737    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-config-volume\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.545310 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.545260    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.545368 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:28:11.545311    2573 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 05 06:28:11.545425 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:28:11.545367    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-main-tls podName:a94c0af7-99fc-443c-b161-17c830e5e131 nodeName:}" failed. No retries permitted until 2026-09-05 06:28:12.045349501 +0000 UTC m=+140.413396693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131") : secret "alertmanager-main-tls" not found
Sep 05 06:28:11.546842 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:28:11.545602    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-trusted-ca-bundle podName:a94c0af7-99fc-443c-b161-17c830e5e131 nodeName:}" failed. No retries permitted until 2026-09-05 06:28:12.04558466 +0000 UTC m=+140.413631863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131") : configmap references non-existent config key: ca-bundle.crt
Sep 05 06:28:11.546842 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.546278    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.549889 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.549844    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.550747 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.550299    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.550747 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.550421    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-web-config\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.550747 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.550691    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-config-volume\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.550972 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.550816    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-config-out\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.551707 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.551685    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.551870 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.551824    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.552394 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.552371    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.554159 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.554131    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zbfm\" (UniqueName: \"kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-kube-api-access-6zbfm\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:11.723411 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.723382    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:11.822338 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.822300    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ln2dx" event={"ID":"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9","Type":"ContainerStarted","Data":"4b359038904db222ef87b2e3abbe0810ea6d4f671fb1bcabe7de7d2eed75d04b"}
Sep 05 06:28:11.824469 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.824445    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:28:11.824631 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.824510    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-4n587" event={"ID":"aa4da948-54e4-40a2-8c68-e6a250c5f2be","Type":"ContainerStarted","Data":"e4a31f044597657cf790ef3df7c525ac91c9a5d5101d6293790bb59d9b35bb39"}
Sep 05 06:28:11.824785 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.824758    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:28:11.864132 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:11.864072    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-4n587" podStartSLOduration=42.019699452 podStartE2EDuration="43.864054464s" totalImagesPullingTime="1.844355012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:11.863098905 +0000 UTC m=+140.231146107" watchObservedRunningTime="2026-09-05 06:28:11.864054464 +0000 UTC m=+140.232101652"
Sep 05 06:28:12.049105 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.049065    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:12.049330 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.049158    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:12.049951 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.049930    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:12.051562 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.051544    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:12.079746 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.079596    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-4n587"
Sep 05 06:28:12.273247 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.273205    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-rrktx"]
Sep 05 06:28:12.276434 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.276412    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-rrktx"
Sep 05 06:28:12.279033 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.279013    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 05 06:28:12.279245 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.279230    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hl7qn\""
Sep 05 06:28:12.279613 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.279591    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 05 06:28:12.287987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.287965    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-rrktx"]
Sep 05 06:28:12.296029 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.296003    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:12.319448 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.319371    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:28:12.351395 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.351360    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmxnn\" (UniqueName: \"kubernetes.io/projected/574cc721-7a78-4807-8f0b-c9268affb086-kube-api-access-lmxnn\") pod \"downloads-85d98c998f-rrktx\" (UID: \"574cc721-7a78-4807-8f0b-c9268affb086\") " pod="openshift-console/downloads-85d98c998f-rrktx"
Sep 05 06:28:12.370075 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.369714    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:12.451759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.451723    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:12.451928 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.451833    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmxnn\" (UniqueName: \"kubernetes.io/projected/574cc721-7a78-4807-8f0b-c9268affb086-kube-api-access-lmxnn\") pod \"downloads-85d98c998f-rrktx\" (UID: \"574cc721-7a78-4807-8f0b-c9268affb086\") " pod="openshift-console/downloads-85d98c998f-rrktx"
Sep 05 06:28:12.461518 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.461495    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmxnn\" (UniqueName: \"kubernetes.io/projected/574cc721-7a78-4807-8f0b-c9268affb086-kube-api-access-lmxnn\") pod \"downloads-85d98c998f-rrktx\" (UID: \"574cc721-7a78-4807-8f0b-c9268affb086\") " pod="openshift-console/downloads-85d98c998f-rrktx"
Sep 05 06:28:12.581943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.581863    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:12.585124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.585107    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-rrktx"
Sep 05 06:28:12.610903 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.610842    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 06:28:12.613961 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:28:12.613927    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda94c0af7_99fc_443c_b161_17c830e5e131.slice/crio-55d470b7dcd0d68134008e3572dfaebb8d66b3b6ff578b7597921a40778df7dd WatchSource:0}: Error finding container 55d470b7dcd0d68134008e3572dfaebb8d66b3b6ff578b7597921a40778df7dd: Status 404 returned error can't find the container with id 55d470b7dcd0d68134008e3572dfaebb8d66b3b6ff578b7597921a40778df7dd
Sep 05 06:28:12.829929 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.829891    2573 generic.go:356] "Generic (PLEG): container finished" podID="16ede8b1-46ac-4eb5-811a-bf567cd0dbf9" containerID="4b359038904db222ef87b2e3abbe0810ea6d4f671fb1bcabe7de7d2eed75d04b" exitCode=0
Sep 05 06:28:12.830081 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.829977    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ln2dx" event={"ID":"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9","Type":"ContainerDied","Data":"4b359038904db222ef87b2e3abbe0810ea6d4f671fb1bcabe7de7d2eed75d04b"}
Sep 05 06:28:12.831247 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.831229    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerStarted","Data":"55d470b7dcd0d68134008e3572dfaebb8d66b3b6ff578b7597921a40778df7dd"}
Sep 05 06:28:12.861585 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.861559    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-rrktx"]
Sep 05 06:28:12.871003 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:28:12.870978    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod574cc721_7a78_4807_8f0b_c9268affb086.slice/crio-2ff2fa724ed1b7e86ca8fac749423dd80c98b94b381d001a32a9cfbdf11582ef WatchSource:0}: Error finding container 2ff2fa724ed1b7e86ca8fac749423dd80c98b94b381d001a32a9cfbdf11582ef: Status 404 returned error can't find the container with id 2ff2fa724ed1b7e86ca8fac749423dd80c98b94b381d001a32a9cfbdf11582ef
Sep 05 06:28:12.949226 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:12.949195    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:13.017081 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:13.017049    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:13.325154 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:13.325115    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:13.425380 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:13.425305    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:13.782046 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:13.782019    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:13.838032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:13.837997    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ln2dx" event={"ID":"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9","Type":"ContainerStarted","Data":"515b667ebfedbc651c7337dab731490ff6f236f699a055454516beef99461995"}
Sep 05 06:28:13.838195 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:13.838063    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ln2dx" event={"ID":"16ede8b1-46ac-4eb5-811a-bf567cd0dbf9","Type":"ContainerStarted","Data":"5148ccf1633d8eb38a65f3be643933cf4109d5f5b4e377634e89d5846b6c6351"}
Sep 05 06:28:13.840355 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:13.840273    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerStarted","Data":"f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352"}
Sep 05 06:28:13.842386 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:13.842360    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-rrktx" event={"ID":"574cc721-7a78-4807-8f0b-c9268affb086","Type":"ContainerStarted","Data":"2ff2fa724ed1b7e86ca8fac749423dd80c98b94b381d001a32a9cfbdf11582ef"}
Sep 05 06:28:13.850326 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:13.850298    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:13.861493 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:13.861436    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ln2dx" podStartSLOduration=2.8003186319999998 podStartE2EDuration="3.861415431s" totalImagesPullingTime="999.431898ms" totalInitContainerRuntime="61.664901ms" isStatefulPod=true podCreationTimestamp="2026-09-05 06:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:13.859144687 +0000 UTC m=+142.227191895" watchObservedRunningTime="2026-09-05 06:28:13.861415431 +0000 UTC m=+142.229462640"
Sep 05 06:28:14.163470 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:14.163436    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:14.262400 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:14.262317    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:14.834711 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:14.834679    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:14.847985 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:14.847947    2573 generic.go:356] "Generic (PLEG): container finished" podID="a94c0af7-99fc-443c-b161-17c830e5e131" containerID="f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352" exitCode=0
Sep 05 06:28:14.848144 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:14.848032    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerDied","Data":"f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352"}
Sep 05 06:28:14.987822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:14.987775    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:15.101949 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.101809    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:15.199384 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.199340    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:15.464919 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.464759    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:15.558464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.558426    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"]
Sep 05 06:28:15.562495 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.562466    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.566468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.566444    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 05 06:28:15.566632 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.566531    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 05 06:28:15.566706 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.566628    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 05 06:28:15.566791 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.566531    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 05 06:28:15.566923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.566580    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gm67z\""
Sep 05 06:28:15.566923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.566577    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 05 06:28:15.571767 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.571745    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 05 06:28:15.580265 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.580184    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"]
Sep 05 06:28:15.684149 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.684101    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65681bbd-95a6-4f05-9231-a98ad19c022f-serving-certs-ca-bundle\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.684346 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.684169    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-secret-telemeter-client\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.684346 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.684220    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-federate-client-tls\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.684346 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.684273    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5t5j\" (UniqueName: \"kubernetes.io/projected/65681bbd-95a6-4f05-9231-a98ad19c022f-kube-api-access-w5t5j\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.684346 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.684307    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-telemeter-client-tls\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.684346 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.684342    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.684603 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.684384    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65681bbd-95a6-4f05-9231-a98ad19c022f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.684603 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.684497    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65681bbd-95a6-4f05-9231-a98ad19c022f-metrics-client-ca\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.785276 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.785236    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65681bbd-95a6-4f05-9231-a98ad19c022f-metrics-client-ca\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.785464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.785295    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65681bbd-95a6-4f05-9231-a98ad19c022f-serving-certs-ca-bundle\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.785464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.785341    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-secret-telemeter-client\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.785464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.785384    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-federate-client-tls\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.785464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.785424    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5t5j\" (UniqueName: \"kubernetes.io/projected/65681bbd-95a6-4f05-9231-a98ad19c022f-kube-api-access-w5t5j\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.785464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.785458    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-telemeter-client-tls\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.786342 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.785915    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.786342 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.785977    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65681bbd-95a6-4f05-9231-a98ad19c022f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.786342 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.786107    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65681bbd-95a6-4f05-9231-a98ad19c022f-metrics-client-ca\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.786342 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.786177    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65681bbd-95a6-4f05-9231-a98ad19c022f-serving-certs-ca-bundle\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.786831 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.786807    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65681bbd-95a6-4f05-9231-a98ad19c022f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.788531 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.788507    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-federate-client-tls\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.788635 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.788583    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-secret-telemeter-client\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.788688 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.788670    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-telemeter-client-tls\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.789131 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.789071    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65681bbd-95a6-4f05-9231-a98ad19c022f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.796096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.796071    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5t5j\" (UniqueName: \"kubernetes.io/projected/65681bbd-95a6-4f05-9231-a98ad19c022f-kube-api-access-w5t5j\") pod \"telemeter-client-9bbfdb7c9-4tsjd\" (UID: \"65681bbd-95a6-4f05-9231-a98ad19c022f\") " pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:15.874779 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:15.874736    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"
Sep 05 06:28:16.463025 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.462880    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd"]
Sep 05 06:28:16.464446 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:28:16.464303    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65681bbd_95a6_4f05_9231_a98ad19c022f.slice/crio-73f6680c26a9354e97f913927b9a7ce7b2bacc08e090af27ac3be938c0dfe001 WatchSource:0}: Error finding container 73f6680c26a9354e97f913927b9a7ce7b2bacc08e090af27ac3be938c0dfe001: Status 404 returned error can't find the container with id 73f6680c26a9354e97f913927b9a7ce7b2bacc08e090af27ac3be938c0dfe001
Sep 05 06:28:16.601838 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.601804    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:16.669326 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.669283    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 06:28:16.675200 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.675165    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.676310 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.676140    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:16.678489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.678325    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 05 06:28:16.678489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.678334    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 05 06:28:16.678489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.678419    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 05 06:28:16.678881 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.678863    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 05 06:28:16.679003 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.678934    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 05 06:28:16.682085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.679182    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c5eckqvfsarlk\""
Sep 05 06:28:16.682085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.679328    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 05 06:28:16.682085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.679401    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xnrfq\""
Sep 05 06:28:16.682085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.679627    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 05 06:28:16.682085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.679961    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 05 06:28:16.682085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.680485    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 05 06:28:16.682085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.680689    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 05 06:28:16.682085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.680954    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 05 06:28:16.682085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.681777    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 05 06:28:16.682539 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.682095    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 05 06:28:16.684226 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.684110    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 05 06:28:16.685557 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.685535    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 05 06:28:16.698440 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.698407    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 06:28:16.751423 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.751356    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796234    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796336    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796368    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796386    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796401    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796418    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796472    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796510    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796536    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796553    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796569    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796596    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.796670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796647    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.797466 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796698    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnb7s\" (UniqueName: \"kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-kube-api-access-fnb7s\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.797466 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796795    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.797466 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796841    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.797466 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796900    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.797466 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796933    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.797466 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796966    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.797466 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.796990    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.860927 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.860892    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerStarted","Data":"07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709"}
Sep 05 06:28:16.861101 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.860978    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerStarted","Data":"fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7"}
Sep 05 06:28:16.861101 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.861036    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerStarted","Data":"09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6"}
Sep 05 06:28:16.861101 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.861074    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerStarted","Data":"f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8"}
Sep 05 06:28:16.861268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.861110    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerStarted","Data":"9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7"}
Sep 05 06:28:16.862379 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.862349    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd" event={"ID":"65681bbd-95a6-4f05-9231-a98ad19c022f","Type":"ContainerStarted","Data":"73f6680c26a9354e97f913927b9a7ce7b2bacc08e090af27ac3be938c0dfe001"}
Sep 05 06:28:16.898186 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898147    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898197    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898224    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898248    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898269    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898324    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898366    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898407    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898434    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898462    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898490    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898512    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898539    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnb7s\" (UniqueName: \"kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-kube-api-access-fnb7s\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898574    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.898592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898599    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.899185 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898635    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.899185 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898682    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.899185 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898731    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.899185 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898763    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.899185 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898801    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.899185 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.898969    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.899914 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.899498    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.900090 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.900063    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.901074 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.901046    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.904380 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.901348    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.904380 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.902931    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.904380 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.904328    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.906572 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.906532    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.907061 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.907037    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.907833 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.907789    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.908092 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.908067    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.909104 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.908648    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.909104 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.908831    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.909247 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.909100    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.909382 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.909303    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.909669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.909647    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.910731 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.910675    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.911034 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.911013    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.911122 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.911108    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.912665 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.912617    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnb7s\" (UniqueName: \"kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-kube-api-access-fnb7s\") pod \"prometheus-k8s-0\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:16.989346 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:16.989238    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:17.295488 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:17.295459    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 06:28:17.452343 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:28:17.452305    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod437bcc3b_1b62_4905_bb17_09a1f31a89f8.slice/crio-09b4e77361058afe3b8077ebfe1b8ffe80757bf3464f0413b9017785e363e71c WatchSource:0}: Error finding container 09b4e77361058afe3b8077ebfe1b8ffe80757bf3464f0413b9017785e363e71c: Status 404 returned error can't find the container with id 09b4e77361058afe3b8077ebfe1b8ffe80757bf3464f0413b9017785e363e71c
Sep 05 06:28:17.878831 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:17.878734    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerStarted","Data":"172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e"}
Sep 05 06:28:17.880870 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:17.880829    2573 generic.go:356] "Generic (PLEG): container finished" podID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerID="787f2e97c5de0c0ddb61133a994be02ad3307d6a5211b2d2a66cbce480432eaa" exitCode=0
Sep 05 06:28:17.881009 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:17.880911    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerDied","Data":"787f2e97c5de0c0ddb61133a994be02ad3307d6a5211b2d2a66cbce480432eaa"}
Sep 05 06:28:17.881078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:17.881011    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerStarted","Data":"09b4e77361058afe3b8077ebfe1b8ffe80757bf3464f0413b9017785e363e71c"}
Sep 05 06:28:17.909950 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:17.909895    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.5030576829999998 podStartE2EDuration="6.909876036s" totalImagesPullingTime="3.386048106s" totalInitContainerRuntime="20.770247ms" isStatefulPod=false podCreationTimestamp="2026-09-05 06:28:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:17.908664295 +0000 UTC m=+146.276711533" watchObservedRunningTime="2026-09-05 06:28:17.909876036 +0000 UTC m=+146.277923247"
Sep 05 06:28:18.890073 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:18.890034    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd" event={"ID":"65681bbd-95a6-4f05-9231-a98ad19c022f","Type":"ContainerStarted","Data":"6dedb147eca68065b24dd8dc89e700a47d2dcfd9da22460ab53e3c52fabdf3f6"}
Sep 05 06:28:18.890263 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:18.890107    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd" event={"ID":"65681bbd-95a6-4f05-9231-a98ad19c022f","Type":"ContainerStarted","Data":"108fbf632b0671f60532df249c9dd45df84d8e2a886b1af0bfa88dc9ec2bad01"}
Sep 05 06:28:18.890263 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:18.890139    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd" event={"ID":"65681bbd-95a6-4f05-9231-a98ad19c022f","Type":"ContainerStarted","Data":"cf40a02c3f7935c139206b39d493eb08dd5aff981e60cfe157083cd14f20512b"}
Sep 05 06:28:18.918271 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:18.918164    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9bbfdb7c9-4tsjd" podStartSLOduration=1.9121727549999998 podStartE2EDuration="3.918137903s" totalImagesPullingTime="2.005965148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:18.914697782 +0000 UTC m=+147.282744994" watchObservedRunningTime="2026-09-05 06:28:18.918137903 +0000 UTC m=+147.286185114"
Sep 05 06:28:20.729891 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:20.729825    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bc8d97d45-s96zd"]
Sep 05 06:28:20.730316 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:28:20.730216    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd" podUID="dbf8d038-21ba-42eb-aac4-7b16c7eae743"
Sep 05 06:28:20.844488 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:20.844448    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:20.899731 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:20.899641    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:28:20.907980 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:20.907947    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:28:20.909038 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:20.909009    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:21.049091 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.049047    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-image-registry-private-configuration\") pod \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") "
Sep 05 06:28:21.049287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.049108    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-certificates\") pod \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") "
Sep 05 06:28:21.049287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.049148    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-bound-sa-token\") pod \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") "
Sep 05 06:28:21.049287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.049190    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-installation-pull-secrets\") pod \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") "
Sep 05 06:28:21.049287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.049215    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mwp7\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-kube-api-access-2mwp7\") pod \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") "
Sep 05 06:28:21.049287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.049251    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-trusted-ca\") pod \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") "
Sep 05 06:28:21.049534 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.049351    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbf8d038-21ba-42eb-aac4-7b16c7eae743-ca-trust-extracted\") pod \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\" (UID: \"dbf8d038-21ba-42eb-aac4-7b16c7eae743\") "
Sep 05 06:28:21.049534 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.049490    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "dbf8d038-21ba-42eb-aac4-7b16c7eae743" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:28:21.049761 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.049741    2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-certificates\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:28:21.050029 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.050006    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf8d038-21ba-42eb-aac4-7b16c7eae743-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "dbf8d038-21ba-42eb-aac4-7b16c7eae743" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:28:21.050243 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.050183    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "dbf8d038-21ba-42eb-aac4-7b16c7eae743" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:28:21.052410 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.052379    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-kube-api-access-2mwp7" (OuterVolumeSpecName: "kube-api-access-2mwp7") pod "dbf8d038-21ba-42eb-aac4-7b16c7eae743" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743"). InnerVolumeSpecName "kube-api-access-2mwp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:28:21.052525 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.052417    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "dbf8d038-21ba-42eb-aac4-7b16c7eae743" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:28:21.052919 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.052892    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "dbf8d038-21ba-42eb-aac4-7b16c7eae743" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:28:21.053447 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.053421    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "dbf8d038-21ba-42eb-aac4-7b16c7eae743" (UID: "dbf8d038-21ba-42eb-aac4-7b16c7eae743"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:28:21.151222 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.151184    2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbf8d038-21ba-42eb-aac4-7b16c7eae743-ca-trust-extracted\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:28:21.151222 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.151221    2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-image-registry-private-configuration\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:28:21.151427 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.151234    2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-bound-sa-token\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:28:21.151427 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.151246    2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbf8d038-21ba-42eb-aac4-7b16c7eae743-installation-pull-secrets\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:28:21.151427 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.151259    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mwp7\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-kube-api-access-2mwp7\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:28:21.151427 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.151270    2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbf8d038-21ba-42eb-aac4-7b16c7eae743-trusted-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:28:21.911429 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.911401    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bc8d97d45-s96zd"
Sep 05 06:28:21.953160 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.953125    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bc8d97d45-s96zd"]
Sep 05 06:28:21.956558 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:21.956529    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6bc8d97d45-s96zd"]
Sep 05 06:28:22.062317 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:22.062252    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbf8d038-21ba-42eb-aac4-7b16c7eae743-registry-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:28:22.156318 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:22.156283    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbf8d038-21ba-42eb-aac4-7b16c7eae743" path="/var/lib/kubelet/pods/dbf8d038-21ba-42eb-aac4-7b16c7eae743/volumes"
Sep 05 06:28:22.919824 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:22.919788    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerStarted","Data":"87c3ffa7895742714b15f4956bac54469f9aaf7d351a2711c5b2c4b579129fcb"}
Sep 05 06:28:22.920226 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:22.919888    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerStarted","Data":"211c266e07cd96f437c9c635cd3b04d74eb073cf5c2831a9476eff728481a7db"}
Sep 05 06:28:23.993513 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:23.993475    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:24.093648 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:24.093610    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:25.839322 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:25.839267    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:25.905316 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:25.905281    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:27.181963 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:27.181923    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:27.949090 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:27.949049    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:28.017052 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:28.017010    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:29.029213 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:28:29.029161    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-xpm9h" podUID="ae33df4f-832d-4ee1-83b7-798f60f540d8"
Sep 05 06:28:29.379500 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:29.379420    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:29.480213 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:29.480142    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:29.952025 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:29.951994    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:28:30.991554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:30.991502    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:31.055899 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.055845    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:31.660486 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.660460    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:31.762468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.762431    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:31.962578 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.962536    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-rrktx" event={"ID":"574cc721-7a78-4807-8f0b-c9268affb086","Type":"ContainerStarted","Data":"0cae5e5803ef7322b3aa88253cf6310fbbd8f8b908156247daa9e147f44a815e"}
Sep 05 06:28:31.962751 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.962719    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-rrktx"
Sep 05 06:28:31.964326 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.964302    2573 patch_prober.go:28] interesting pod/downloads-85d98c998f-rrktx container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.18:8080/\": dial tcp 10.132.0.18:8080: connect: connection refused" start-of-body=
Sep 05 06:28:31.964449 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.964353    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-85d98c998f-rrktx" podUID="574cc721-7a78-4807-8f0b-c9268affb086" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.18:8080/\": dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 05 06:28:31.966495 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.966462    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerStarted","Data":"6922b1f5ba4ce6dcda56f6911d08234676bf91a79908cc45fab3a09abe02d5ea"}
Sep 05 06:28:31.966608 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.966536    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerStarted","Data":"0439aa835a02bca8e0329c9b86b7df75967b50a7910e6bfa2ced01acb695b170"}
Sep 05 06:28:31.966608 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.966588    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerStarted","Data":"5ce3a944e99b3fe32c3208fa4ab5a1a358a405af4d1a6ce730c40d74fbaabfca"}
Sep 05 06:28:31.966701 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.966657    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerStarted","Data":"9dd6707a9493cc3bde178116948d773de86d231306ad04a055628303ff6e0827"}
Sep 05 06:28:31.983838 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.983785    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-rrktx" podStartSLOduration=1.265207615 podStartE2EDuration="19.983764071s" totalImagesPullingTime="18.718556456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:28:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:31.981218086 +0000 UTC m=+160.349265323" watchObservedRunningTime="2026-09-05 06:28:31.983764071 +0000 UTC m=+160.351811283"
Sep 05 06:28:31.990231 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:31.990199    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:28:32.015873 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:32.015802    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.502923272 podStartE2EDuration="16.015781753s" totalImagesPullingTime="13.489128178s" totalInitContainerRuntime="23.730303ms" isStatefulPod=false podCreationTimestamp="2026-09-05 06:28:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:32.014132966 +0000 UTC m=+160.382180177" watchObservedRunningTime="2026-09-05 06:28:32.015781753 +0000 UTC m=+160.383828965"
Sep 05 06:28:32.158158 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:32.158074    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:32.259996 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:32.259957    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:32.981869 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:32.981825    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-rrktx"
Sep 05 06:28:33.890338 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:33.890292    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:28:33.892809 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:33.892782    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ae33df4f-832d-4ee1-83b7-798f60f540d8-metrics-tls\") pod \"dns-default-xpm9h\" (UID: \"ae33df4f-832d-4ee1-83b7-798f60f540d8\") " pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:28:34.155226 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:34.155155    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v76tl\""
Sep 05 06:28:34.163473 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:34.163443    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:28:34.485211 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:34.485130    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xpm9h"]
Sep 05 06:28:34.489143 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:28:34.489101    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae33df4f_832d_4ee1_83b7_798f60f540d8.slice/crio-9cc41fc2fbf8ec3b3a0afc0d32a6b590735afc2c1ceaa55feefa4827c0abc98f WatchSource:0}: Error finding container 9cc41fc2fbf8ec3b3a0afc0d32a6b590735afc2c1ceaa55feefa4827c0abc98f: Status 404 returned error can't find the container with id 9cc41fc2fbf8ec3b3a0afc0d32a6b590735afc2c1ceaa55feefa4827c0abc98f
Sep 05 06:28:34.985474 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:34.985440    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xpm9h" event={"ID":"ae33df4f-832d-4ee1-83b7-798f60f540d8","Type":"ContainerStarted","Data":"9cc41fc2fbf8ec3b3a0afc0d32a6b590735afc2c1ceaa55feefa4827c0abc98f"}
Sep 05 06:28:35.673197 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:35.673161    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:35.774173 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:35.774135    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:36.201194 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:36.201134    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:36.301858 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:36.301797    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:37.594466 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:37.594381    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:37.654518 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:37.654481    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:38.003227 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:38.003188    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xpm9h" event={"ID":"ae33df4f-832d-4ee1-83b7-798f60f540d8","Type":"ContainerStarted","Data":"25530d706276df3c102bb41a96b8c4e04ed3e51165ef9adee32e976c1b64da57"}
Sep 05 06:28:38.003418 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:38.003258    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xpm9h" event={"ID":"ae33df4f-832d-4ee1-83b7-798f60f540d8","Type":"ContainerStarted","Data":"35e3c12132c0191f6268ca6fc2a054e2026ca8fcaaf76b4bd55873916ed9d608"}
Sep 05 06:28:38.003418 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:38.003291    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:28:38.505584 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:38.505543    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:38.606474 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:38.606438    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:40.202535 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:40.202494    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:40.295875 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:40.295822    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:42.822959 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:42.822925    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:42.883913 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:42.883875    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:43.806973 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:43.806938    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:46.648065 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:46.648031    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:46.704098 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:46.704064    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:48.010682 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:48.010652    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xpm9h"
Sep 05 06:28:48.036727 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:48.036668    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xpm9h" podStartSLOduration=140.511267845 podStartE2EDuration="2m23.036651408s" totalImagesPullingTime="2.525383563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:38.022039174 +0000 UTC m=+166.390086384" watchObservedRunningTime="2026-09-05 06:28:48.036651408 +0000 UTC m=+176.404698641"
Sep 05 06:28:48.768824 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:48.768791    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-llbmp_5d2c676f-9881-4180-9aa4-2cc3757adfa0/serve-healthcheck-canary/0.log"
Sep 05 06:28:57.184819 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:57.184783    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:28:57.252203 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:28:57.252163    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:01.651772 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:01.651731    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:01.720513 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:01.720475    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:08.224429 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:08.224392    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:08.320692 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:08.320654    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:08.385171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:08.385144    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:16.990269 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:16.990225    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:17.044302 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:17.044274    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:17.175162 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:17.175135    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:26.493888 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:26.490496    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:26.590540 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:26.590501    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:29.383038 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:29.383002    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:29.517108 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:29.517074    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:29.582446 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:29.582415    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:30.114580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:30.114547    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:30.176816 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:30.176781    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:30.827331 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:30.827292    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 06:29:30.827801 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:30.827706    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="alertmanager" containerID="cri-o://9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7" gracePeriod=120
Sep 05 06:29:30.827801 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:30.827779    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy-web" containerID="cri-o://09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6" gracePeriod=120
Sep 05 06:29:30.827961 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:30.827804    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="prom-label-proxy" containerID="cri-o://172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e" gracePeriod=120
Sep 05 06:29:30.827961 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:30.827813    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="config-reloader" containerID="cri-o://f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8" gracePeriod=120
Sep 05 06:29:30.827961 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:30.827779    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy" containerID="cri-o://fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7" gracePeriod=120
Sep 05 06:29:30.828083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:30.827779    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy-metric" containerID="cri-o://07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709" gracePeriod=120
Sep 05 06:29:31.215519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:31.215436    2573 generic.go:356] "Generic (PLEG): container finished" podID="a94c0af7-99fc-443c-b161-17c830e5e131" containerID="172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e" exitCode=0
Sep 05 06:29:31.215519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:31.215458    2573 generic.go:356] "Generic (PLEG): container finished" podID="a94c0af7-99fc-443c-b161-17c830e5e131" containerID="fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7" exitCode=0
Sep 05 06:29:31.215519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:31.215464    2573 generic.go:356] "Generic (PLEG): container finished" podID="a94c0af7-99fc-443c-b161-17c830e5e131" containerID="f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8" exitCode=0
Sep 05 06:29:31.215519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:31.215470    2573 generic.go:356] "Generic (PLEG): container finished" podID="a94c0af7-99fc-443c-b161-17c830e5e131" containerID="9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7" exitCode=0
Sep 05 06:29:31.215749 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:31.215513    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerDied","Data":"172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e"}
Sep 05 06:29:31.215749 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:31.215581    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerDied","Data":"fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7"}
Sep 05 06:29:31.215749 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:31.215610    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerDied","Data":"f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8"}
Sep 05 06:29:31.215749 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:31.215633    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerDied","Data":"9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7"}
Sep 05 06:29:32.073263 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.073233    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.101145 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101114    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101309 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101162    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-trusted-ca-bundle\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101309 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101183    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-config-volume\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101309 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101222    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101309 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101259    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zbfm\" (UniqueName: \"kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-kube-api-access-6zbfm\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101307    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-web\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101333    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-main-tls\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101359    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-config-out\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101392    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-main-db\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101422    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-tls-assets\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101450    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-cluster-tls-config\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101488    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-web-config\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101843 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101539    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-metrics-client-ca\") pod \"a94c0af7-99fc-443c-b161-17c830e5e131\" (UID: \"a94c0af7-99fc-443c-b161-17c830e5e131\") "
Sep 05 06:29:32.101843 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101631    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:29:32.101843 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.101795    2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.102767 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.102720    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:29:32.103172 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.103079    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:29:32.105140 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.105101    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:32.106651 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.106621    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-kube-api-access-6zbfm" (OuterVolumeSpecName: "kube-api-access-6zbfm") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "kube-api-access-6zbfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:29:32.106797 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.106646    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:29:32.106797 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.106644    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:32.106989 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.106863    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:32.107462 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.107428    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-config-volume" (OuterVolumeSpecName: "config-volume") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:32.107962 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.107928    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-config-out" (OuterVolumeSpecName: "config-out") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:29:32.108059 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.108008    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:32.110889 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.110826    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:32.121969 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.121939    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-web-config" (OuterVolumeSpecName: "web-config") pod "a94c0af7-99fc-443c-b161-17c830e5e131" (UID: "a94c0af7-99fc-443c-b161-17c830e5e131"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:32.203165 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203126    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.203165 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203163    2573 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.203165 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203177    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.203468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203192    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zbfm\" (UniqueName: \"kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-kube-api-access-6zbfm\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.203468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203205    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.203468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203219    2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-secret-alertmanager-main-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.203468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203231    2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-config-out\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.203468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203244    2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a94c0af7-99fc-443c-b161-17c830e5e131-alertmanager-main-db\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.203468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203256    2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a94c0af7-99fc-443c-b161-17c830e5e131-tls-assets\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.203468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203269    2573 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-cluster-tls-config\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.203468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203281    2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a94c0af7-99fc-443c-b161-17c830e5e131-web-config\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.203468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.203296    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a94c0af7-99fc-443c-b161-17c830e5e131-metrics-client-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:32.222951 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.222911    2573 generic.go:356] "Generic (PLEG): container finished" podID="a94c0af7-99fc-443c-b161-17c830e5e131" containerID="07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709" exitCode=0
Sep 05 06:29:32.222951 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.222941    2573 generic.go:356] "Generic (PLEG): container finished" podID="a94c0af7-99fc-443c-b161-17c830e5e131" containerID="09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6" exitCode=0
Sep 05 06:29:32.223139 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.222958    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerDied","Data":"07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709"}
Sep 05 06:29:32.223139 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.222994    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.223139 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.223026    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerDied","Data":"09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6"}
Sep 05 06:29:32.223139 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.223057    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a94c0af7-99fc-443c-b161-17c830e5e131","Type":"ContainerDied","Data":"55d470b7dcd0d68134008e3572dfaebb8d66b3b6ff578b7597921a40778df7dd"}
Sep 05 06:29:32.223139 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.223088    2573 scope.go:117] "RemoveContainer" containerID="172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e"
Sep 05 06:29:32.231159 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.231140    2573 scope.go:117] "RemoveContainer" containerID="07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709"
Sep 05 06:29:32.238516 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.238498    2573 scope.go:117] "RemoveContainer" containerID="fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7"
Sep 05 06:29:32.245325 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.245298    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 06:29:32.246528 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.246512    2573 scope.go:117] "RemoveContainer" containerID="09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6"
Sep 05 06:29:32.251462 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.251438    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 06:29:32.254591 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.254574    2573 scope.go:117] "RemoveContainer" containerID="f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8"
Sep 05 06:29:32.261652 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.261633    2573 scope.go:117] "RemoveContainer" containerID="9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7"
Sep 05 06:29:32.269285 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.269262    2573 scope.go:117] "RemoveContainer" containerID="f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352"
Sep 05 06:29:32.276916 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.276896    2573 scope.go:117] "RemoveContainer" containerID="172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e"
Sep 05 06:29:32.277225 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:29:32.277196    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e\": container with ID starting with 172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e not found: ID does not exist" containerID="172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e"
Sep 05 06:29:32.277322 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.277232    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e"} err="failed to get container status \"172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e\": rpc error: code = NotFound desc = could not find container \"172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e\": container with ID starting with 172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e not found: ID does not exist"
Sep 05 06:29:32.277322 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.277273    2573 scope.go:117] "RemoveContainer" containerID="07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709"
Sep 05 06:29:32.277595 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:29:32.277540    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709\": container with ID starting with 07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709 not found: ID does not exist" containerID="07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709"
Sep 05 06:29:32.277595 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.277568    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709"} err="failed to get container status \"07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709\": rpc error: code = NotFound desc = could not find container \"07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709\": container with ID starting with 07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709 not found: ID does not exist"
Sep 05 06:29:32.277595 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.277591    2573 scope.go:117] "RemoveContainer" containerID="fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7"
Sep 05 06:29:32.278062 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:29:32.278023    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7\": container with ID starting with fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7 not found: ID does not exist" containerID="fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7"
Sep 05 06:29:32.278174 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.278060    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7"} err="failed to get container status \"fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7\": rpc error: code = NotFound desc = could not find container \"fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7\": container with ID starting with fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7 not found: ID does not exist"
Sep 05 06:29:32.278174 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.278087    2573 scope.go:117] "RemoveContainer" containerID="09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6"
Sep 05 06:29:32.278586 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:29:32.278561    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6\": container with ID starting with 09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6 not found: ID does not exist" containerID="09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6"
Sep 05 06:29:32.278663 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.278592    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6"} err="failed to get container status \"09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6\": rpc error: code = NotFound desc = could not find container \"09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6\": container with ID starting with 09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6 not found: ID does not exist"
Sep 05 06:29:32.278663 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.278608    2573 scope.go:117] "RemoveContainer" containerID="f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8"
Sep 05 06:29:32.278950 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:29:32.278930    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8\": container with ID starting with f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8 not found: ID does not exist" containerID="f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8"
Sep 05 06:29:32.279029 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.278958    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8"} err="failed to get container status \"f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8\": rpc error: code = NotFound desc = could not find container \"f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8\": container with ID starting with f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8 not found: ID does not exist"
Sep 05 06:29:32.279029 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.278971    2573 scope.go:117] "RemoveContainer" containerID="9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7"
Sep 05 06:29:32.279288 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:29:32.279267    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7\": container with ID starting with 9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7 not found: ID does not exist" containerID="9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7"
Sep 05 06:29:32.279363 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.279294    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7"} err="failed to get container status \"9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7\": rpc error: code = NotFound desc = could not find container \"9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7\": container with ID starting with 9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7 not found: ID does not exist"
Sep 05 06:29:32.279363 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.279315    2573 scope.go:117] "RemoveContainer" containerID="f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352"
Sep 05 06:29:32.279621 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:29:32.279597    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352\": container with ID starting with f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352 not found: ID does not exist" containerID="f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352"
Sep 05 06:29:32.279681 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.279627    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352"} err="failed to get container status \"f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352\": rpc error: code = NotFound desc = could not find container \"f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352\": container with ID starting with f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352 not found: ID does not exist"
Sep 05 06:29:32.279681 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.279648    2573 scope.go:117] "RemoveContainer" containerID="172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e"
Sep 05 06:29:32.279931 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.279911    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e"} err="failed to get container status \"172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e\": rpc error: code = NotFound desc = could not find container \"172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e\": container with ID starting with 172c505b4bc7d0d7ec6724fe60a4f79dc90908a98fffa40de2c74e4aa450221e not found: ID does not exist"
Sep 05 06:29:32.279987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.279933    2573 scope.go:117] "RemoveContainer" containerID="07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709"
Sep 05 06:29:32.280093 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280077    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 06:29:32.280205 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280179    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709"} err="failed to get container status \"07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709\": rpc error: code = NotFound desc = could not find container \"07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709\": container with ID starting with 07b17bbb5fde947a1dc6a2d2eb22f9b392009ade51faecf8cd28ba4dcebcd709 not found: ID does not exist"
Sep 05 06:29:32.280248 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280206    2573 scope.go:117] "RemoveContainer" containerID="fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7"
Sep 05 06:29:32.280387 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280374    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy"
Sep 05 06:29:32.280446 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280389    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy"
Sep 05 06:29:32.280446 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280398    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="alertmanager"
Sep 05 06:29:32.280446 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280404    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="alertmanager"
Sep 05 06:29:32.280446 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280410    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="config-reloader"
Sep 05 06:29:32.280446 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280415    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="config-reloader"
Sep 05 06:29:32.280446 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280422    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy-web"
Sep 05 06:29:32.280446 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280413    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7"} err="failed to get container status \"fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7\": rpc error: code = NotFound desc = could not find container \"fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7\": container with ID starting with fba3f30768171c0a1c5feea007b3de9f840ac9be6ee920a9574fa5fdcf3947f7 not found: ID does not exist"
Sep 05 06:29:32.280446 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280443    2573 scope.go:117] "RemoveContainer" containerID="09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280427    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy-web"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280524    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="prom-label-proxy"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280536    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="prom-label-proxy"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280567    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="init-config-reloader"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280576    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="init-config-reloader"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280592    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy-metric"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280600    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy-metric"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280668    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6"} err="failed to get container status \"09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6\": rpc error: code = NotFound desc = could not find container \"09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6\": container with ID starting with 09475fd71e12605fdb84262221c1aa99f3e9f4cb7965715687d2f8f04e8b98e6 not found: ID does not exist"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280681    2573 scope.go:117] "RemoveContainer" containerID="f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280685    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy-web"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280700    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="config-reloader"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280711    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280722    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="prom-label-proxy"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280730    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="alertmanager"
Sep 05 06:29:32.280822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280740    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" containerName="kube-rbac-proxy-metric"
Sep 05 06:29:32.281606 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280892    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8"} err="failed to get container status \"f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8\": rpc error: code = NotFound desc = could not find container \"f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8\": container with ID starting with f07e17dcbed92dfef6ac50a443dd83f6cd3d473a15b3b7b138a3de9da4abc8a8 not found: ID does not exist"
Sep 05 06:29:32.281606 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.280909    2573 scope.go:117] "RemoveContainer" containerID="9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7"
Sep 05 06:29:32.281606 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.281108    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7"} err="failed to get container status \"9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7\": rpc error: code = NotFound desc = could not find container \"9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7\": container with ID starting with 9f17a6642f3e753070123bde2e07410ac8f00e4b5ee437f5ded5285e2d557ab7 not found: ID does not exist"
Sep 05 06:29:32.281606 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.281125    2573 scope.go:117] "RemoveContainer" containerID="f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352"
Sep 05 06:29:32.281606 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.281328    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352"} err="failed to get container status \"f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352\": rpc error: code = NotFound desc = could not find container \"f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352\": container with ID starting with f618f9548873318ad880ce6d1aba7b450552440b53cfb05ae7aa4f1c194b3352 not found: ID does not exist"
Sep 05 06:29:32.286171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.286149    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.289445 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.289321    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 05 06:29:32.289445 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.289327    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 05 06:29:32.289445 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.289327    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 05 06:29:32.289699 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.289578    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 05 06:29:32.289699 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.289681    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 05 06:29:32.289783 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.289755    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-72zm4\""
Sep 05 06:29:32.289908 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.289888    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 05 06:29:32.290034 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.290015    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 05 06:29:32.290078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.290054    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 05 06:29:32.295312 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.295288    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 05 06:29:32.301822 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.301794    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 06:29:32.405062 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405028    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0319437e-7daa-42ba-935d-683101d2980e-config-out\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405062 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405069    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0319437e-7daa-42ba-935d-683101d2980e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405290 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405092    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgtc8\" (UniqueName: \"kubernetes.io/projected/0319437e-7daa-42ba-935d-683101d2980e-kube-api-access-fgtc8\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405290 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405178    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405290 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405224    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405290 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405243    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405290 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405288    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405496 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405310    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-web-config\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405496 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405338    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405496 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405360    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405496 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405379    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0319437e-7daa-42ba-935d-683101d2980e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405496 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405400    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0319437e-7daa-42ba-935d-683101d2980e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.405496 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.405421    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0319437e-7daa-42ba-935d-683101d2980e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.506707 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.506672    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.506909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.506712    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.506909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.506734    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.506909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.506753    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-web-config\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.506909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.506789    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.506909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.506813    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.506909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.506843    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0319437e-7daa-42ba-935d-683101d2980e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.506909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.506891    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0319437e-7daa-42ba-935d-683101d2980e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.507256 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.506916    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0319437e-7daa-42ba-935d-683101d2980e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.507256 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.506956    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0319437e-7daa-42ba-935d-683101d2980e-config-out\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.507256 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.506992    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0319437e-7daa-42ba-935d-683101d2980e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.507256 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.507020    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgtc8\" (UniqueName: \"kubernetes.io/projected/0319437e-7daa-42ba-935d-683101d2980e-kube-api-access-fgtc8\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.507256 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.507065    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.507511 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.507342    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0319437e-7daa-42ba-935d-683101d2980e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.510080 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.509924    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.510080 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.509924    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-web-config\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.510080 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.509991    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0319437e-7daa-42ba-935d-683101d2980e-config-out\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.510080 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.510007    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.510080 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.510020    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.510400 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.510136    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.510400 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.510196    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.510400 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.510226    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0319437e-7daa-42ba-935d-683101d2980e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.510670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.510648    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0319437e-7daa-42ba-935d-683101d2980e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.510738 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.510649    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0319437e-7daa-42ba-935d-683101d2980e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.511695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.511671    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0319437e-7daa-42ba-935d-683101d2980e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.521355 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.521335    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgtc8\" (UniqueName: \"kubernetes.io/projected/0319437e-7daa-42ba-935d-683101d2980e-kube-api-access-fgtc8\") pod \"alertmanager-main-0\" (UID: \"0319437e-7daa-42ba-935d-683101d2980e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.596550 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.596516    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 05 06:29:32.880273 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:32.880144    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 05 06:29:32.882446 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:29:32.882418    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0319437e_7daa_42ba_935d_683101d2980e.slice/crio-27defd4c70e298c7ef831a6468b23488e3364f5bc6b45df3b49dc6b4e63516a8 WatchSource:0}: Error finding container 27defd4c70e298c7ef831a6468b23488e3364f5bc6b45df3b49dc6b4e63516a8: Status 404 returned error can't find the container with id 27defd4c70e298c7ef831a6468b23488e3364f5bc6b45df3b49dc6b4e63516a8
Sep 05 06:29:33.229508 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:33.229414    2573 generic.go:356] "Generic (PLEG): container finished" podID="0319437e-7daa-42ba-935d-683101d2980e" containerID="1e13693baadf457f9f1dc7f8348e949786a42ab53f939f0a9ccda8ca89de53a3" exitCode=0
Sep 05 06:29:33.229508 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:33.229495    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0319437e-7daa-42ba-935d-683101d2980e","Type":"ContainerDied","Data":"1e13693baadf457f9f1dc7f8348e949786a42ab53f939f0a9ccda8ca89de53a3"}
Sep 05 06:29:33.230044 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:33.229562    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0319437e-7daa-42ba-935d-683101d2980e","Type":"ContainerStarted","Data":"27defd4c70e298c7ef831a6468b23488e3364f5bc6b45df3b49dc6b4e63516a8"}
Sep 05 06:29:34.155845 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:34.155808    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a94c0af7-99fc-443c-b161-17c830e5e131" path="/var/lib/kubelet/pods/a94c0af7-99fc-443c-b161-17c830e5e131/volumes"
Sep 05 06:29:34.236604 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:34.236567    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0319437e-7daa-42ba-935d-683101d2980e","Type":"ContainerStarted","Data":"d6db336d3436d26c3725d04a91b31c2d9324eff3d926a569229c12679afb86f0"}
Sep 05 06:29:34.236965 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:34.236634    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0319437e-7daa-42ba-935d-683101d2980e","Type":"ContainerStarted","Data":"01c745e432f944a5a660388369a562cfb0ef09dc813a2260271d65304eaf41d2"}
Sep 05 06:29:34.236965 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:34.236675    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0319437e-7daa-42ba-935d-683101d2980e","Type":"ContainerStarted","Data":"d2b5581bc814fb92b318bbaadc2d9bbf9cd0758d8633fc6fd612ba56f8abf759"}
Sep 05 06:29:34.236965 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:34.236703    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0319437e-7daa-42ba-935d-683101d2980e","Type":"ContainerStarted","Data":"c79233b19f08eeb393de273d02334ccd0361e95cb2af510a66a1270e010e4369"}
Sep 05 06:29:34.236965 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:34.236730    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0319437e-7daa-42ba-935d-683101d2980e","Type":"ContainerStarted","Data":"4feaee003b7a2155703a07b0835889f6dbd6d3de9916f815570f36d8929868bf"}
Sep 05 06:29:34.236965 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:34.236768    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0319437e-7daa-42ba-935d-683101d2980e","Type":"ContainerStarted","Data":"b72d40146e9cca8063aeb9b67ee33b43d17aa9d2e1460c2c5af7dcb012aafe89"}
Sep 05 06:29:34.264940 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:34.264884    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.248398772 podStartE2EDuration="2.26484694s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.448168ms" isStatefulPod=false podCreationTimestamp="2026-09-05 06:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:29:34.26324709 +0000 UTC m=+222.631294299" watchObservedRunningTime="2026-09-05 06:29:34.26484694 +0000 UTC m=+222.632894149"
Sep 05 06:29:35.077120 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.077089    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 06:29:35.077410 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.077379    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="prometheus" containerID="cri-o://87c3ffa7895742714b15f4956bac54469f9aaf7d351a2711c5b2c4b579129fcb" gracePeriod=600
Sep 05 06:29:35.077410 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.077411    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy-web" containerID="cri-o://5ce3a944e99b3fe32c3208fa4ab5a1a358a405af4d1a6ce730c40d74fbaabfca" gracePeriod=600
Sep 05 06:29:35.077563 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.077477    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy" containerID="cri-o://0439aa835a02bca8e0329c9b86b7df75967b50a7910e6bfa2ced01acb695b170" gracePeriod=600
Sep 05 06:29:35.077563 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.077391    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6922b1f5ba4ce6dcda56f6911d08234676bf91a79908cc45fab3a09abe02d5ea" gracePeriod=600
Sep 05 06:29:35.077662 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.077582    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="thanos-sidecar" containerID="cri-o://9dd6707a9493cc3bde178116948d773de86d231306ad04a055628303ff6e0827" gracePeriod=600
Sep 05 06:29:35.077662 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.077636    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="config-reloader" containerID="cri-o://211c266e07cd96f437c9c635cd3b04d74eb073cf5c2831a9476eff728481a7db" gracePeriod=600
Sep 05 06:29:35.245390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245359    2573 generic.go:356] "Generic (PLEG): container finished" podID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerID="6922b1f5ba4ce6dcda56f6911d08234676bf91a79908cc45fab3a09abe02d5ea" exitCode=0
Sep 05 06:29:35.245390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245384    2573 generic.go:356] "Generic (PLEG): container finished" podID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerID="0439aa835a02bca8e0329c9b86b7df75967b50a7910e6bfa2ced01acb695b170" exitCode=0
Sep 05 06:29:35.245390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245391    2573 generic.go:356] "Generic (PLEG): container finished" podID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerID="5ce3a944e99b3fe32c3208fa4ab5a1a358a405af4d1a6ce730c40d74fbaabfca" exitCode=0
Sep 05 06:29:35.245390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245396    2573 generic.go:356] "Generic (PLEG): container finished" podID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerID="9dd6707a9493cc3bde178116948d773de86d231306ad04a055628303ff6e0827" exitCode=0
Sep 05 06:29:35.245390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245401    2573 generic.go:356] "Generic (PLEG): container finished" podID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerID="211c266e07cd96f437c9c635cd3b04d74eb073cf5c2831a9476eff728481a7db" exitCode=0
Sep 05 06:29:35.245390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245407    2573 generic.go:356] "Generic (PLEG): container finished" podID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerID="87c3ffa7895742714b15f4956bac54469f9aaf7d351a2711c5b2c4b579129fcb" exitCode=0
Sep 05 06:29:35.246082 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245431    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerDied","Data":"6922b1f5ba4ce6dcda56f6911d08234676bf91a79908cc45fab3a09abe02d5ea"}
Sep 05 06:29:35.246082 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245540    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerDied","Data":"0439aa835a02bca8e0329c9b86b7df75967b50a7910e6bfa2ced01acb695b170"}
Sep 05 06:29:35.246082 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245569    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerDied","Data":"5ce3a944e99b3fe32c3208fa4ab5a1a358a405af4d1a6ce730c40d74fbaabfca"}
Sep 05 06:29:35.246082 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245593    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerDied","Data":"9dd6707a9493cc3bde178116948d773de86d231306ad04a055628303ff6e0827"}
Sep 05 06:29:35.246082 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245617    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerDied","Data":"211c266e07cd96f437c9c635cd3b04d74eb073cf5c2831a9476eff728481a7db"}
Sep 05 06:29:35.246082 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.245639    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerDied","Data":"87c3ffa7895742714b15f4956bac54469f9aaf7d351a2711c5b2c4b579129fcb"}
Sep 05 06:29:35.335890 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.335346    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:35.430714 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.430684    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-serving-certs-ca-bundle\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.430714 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.430715    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-kubelet-serving-ca-bundle\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.430987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.430740    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.430987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.430764    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-trusted-ca-bundle\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.430987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.430805    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-grpc-tls\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.430987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.430827    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-web-config\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.430987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.430874    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-kube-rbac-proxy\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.430987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.430975    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431020    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-0\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431053    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-thanos-prometheus-http-client-file\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431095    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-2\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431126    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-tls-assets\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431141    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:29:35.431275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431161    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config-out\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431171    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:29:35.431275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431210    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-db\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431240    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnb7s\" (UniqueName: \"kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-kube-api-access-fnb7s\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431277    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-tls\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431780 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431310    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-metrics-client-certs\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431780 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431350    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-metrics-client-ca\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431780 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431390    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431780 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431419    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-1\") pod \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\" (UID: \"437bcc3b-1b62-4905-bb17-09a1f31a89f8\") "
Sep 05 06:29:35.431780 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431579    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:29:35.432104 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431798    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.432104 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431818    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.432104 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431824    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:29:35.432104 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.431833    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.432298 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.432134    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:29:35.433718 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.433408    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:29:35.434076 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.433919    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:29:35.434158 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.434141    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:29:35.434502 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.434466    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:35.434587 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.434546    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:35.434682 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.434655    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:35.435386 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.435361    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:35.435693 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.435659    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:35.435830 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.435804    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config-out" (OuterVolumeSpecName: "config-out") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:29:35.436647 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.436624    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config" (OuterVolumeSpecName: "config") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:35.436747 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.436710    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-kube-api-access-fnb7s" (OuterVolumeSpecName: "kube-api-access-fnb7s") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "kube-api-access-fnb7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:29:35.436914 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.436887    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:29:35.437020 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.436981    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:35.437020 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.437000    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:35.445536 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.445512    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-web-config" (OuterVolumeSpecName: "web-config") pod "437bcc3b-1b62-4905-bb17-09a1f31a89f8" (UID: "437bcc3b-1b62-4905-bb17-09a1f31a89f8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:29:35.532441 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532398    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532441 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532434    2573 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532441 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532445    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532455    2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-tls-assets\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532465    2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config-out\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532473    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-db\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532481    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnb7s\" (UniqueName: \"kubernetes.io/projected/437bcc3b-1b62-4905-bb17-09a1f31a89f8-kube-api-access-fnb7s\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532489    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532498    2573 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-metrics-client-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532506    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-configmap-metrics-client-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532515    2573 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-config\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532523    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/437bcc3b-1b62-4905-bb17-09a1f31a89f8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532533    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532543    2573 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-grpc-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532550    2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-web-config\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532558    2573 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-kube-rbac-proxy\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:35.532685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:35.532566    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/437bcc3b-1b62-4905-bb17-09a1f31a89f8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:29:36.252905 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.252869    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"437bcc3b-1b62-4905-bb17-09a1f31a89f8","Type":"ContainerDied","Data":"09b4e77361058afe3b8077ebfe1b8ffe80757bf3464f0413b9017785e363e71c"}
Sep 05 06:29:36.253295 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.252937    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.253295 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.252945    2573 scope.go:117] "RemoveContainer" containerID="6922b1f5ba4ce6dcda56f6911d08234676bf91a79908cc45fab3a09abe02d5ea"
Sep 05 06:29:36.260734 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.260716    2573 scope.go:117] "RemoveContainer" containerID="0439aa835a02bca8e0329c9b86b7df75967b50a7910e6bfa2ced01acb695b170"
Sep 05 06:29:36.268147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.268128    2573 scope.go:117] "RemoveContainer" containerID="5ce3a944e99b3fe32c3208fa4ab5a1a358a405af4d1a6ce730c40d74fbaabfca"
Sep 05 06:29:36.274778 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.274760    2573 scope.go:117] "RemoveContainer" containerID="9dd6707a9493cc3bde178116948d773de86d231306ad04a055628303ff6e0827"
Sep 05 06:29:36.278958 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.278935    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 06:29:36.282161 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.282143    2573 scope.go:117] "RemoveContainer" containerID="211c266e07cd96f437c9c635cd3b04d74eb073cf5c2831a9476eff728481a7db"
Sep 05 06:29:36.286236 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.286214    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 06:29:36.289524 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.289507    2573 scope.go:117] "RemoveContainer" containerID="87c3ffa7895742714b15f4956bac54469f9aaf7d351a2711c5b2c4b579129fcb"
Sep 05 06:29:36.296626 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.296609    2573 scope.go:117] "RemoveContainer" containerID="787f2e97c5de0c0ddb61133a994be02ad3307d6a5211b2d2a66cbce480432eaa"
Sep 05 06:29:36.315445 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315411    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 06:29:36.315761 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315743    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="init-config-reloader"
Sep 05 06:29:36.315836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315764    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="init-config-reloader"
Sep 05 06:29:36.315836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315791    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy"
Sep 05 06:29:36.315836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315801    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy"
Sep 05 06:29:36.315836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315813    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy-thanos"
Sep 05 06:29:36.315836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315823    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy-thanos"
Sep 05 06:29:36.315836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315835    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy-web"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315844    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy-web"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315878    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="prometheus"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315886    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="prometheus"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315897    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="config-reloader"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315905    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="config-reloader"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315917    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="thanos-sidecar"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.315926    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="thanos-sidecar"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.316001    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="config-reloader"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.316014    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="prometheus"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.316023    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy-web"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.316032    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="thanos-sidecar"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.316042    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy"
Sep 05 06:29:36.316147 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.316052    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" containerName="kube-rbac-proxy-thanos"
Sep 05 06:29:36.321490 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.321470    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.324175 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.324154    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 05 06:29:36.324307 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.324262    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 05 06:29:36.324307 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.324273    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 05 06:29:36.324715 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.324688    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 05 06:29:36.324715 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.324690    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 05 06:29:36.324959 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.324752    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 05 06:29:36.324959 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.324769    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 05 06:29:36.324959 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.324822    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 05 06:29:36.325125 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.325089    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 05 06:29:36.325356 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.325336    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c5eckqvfsarlk\""
Sep 05 06:29:36.325546 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.325518    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xnrfq\""
Sep 05 06:29:36.325704 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.325685    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 05 06:29:36.327510 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.325844    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 05 06:29:36.327773 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.327752    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 05 06:29:36.327948 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.327928    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 05 06:29:36.329725 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.329700    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 05 06:29:36.330861 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.330831    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 05 06:29:36.335929 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.335904    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 06:29:36.441616 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441516    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.441616 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441559    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.441616 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441582    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.441831 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441647    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.441831 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441695    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd293127-52cd-46ea-aa6d-6c3d8af86732-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.441831 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441726    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.441831 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441760    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.441831 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441813    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441839    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-web-config\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441905    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd293127-52cd-46ea-aa6d-6c3d8af86732-config-out\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441925    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441941    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441960    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.441993    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.442009    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-config\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.442023    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.442073    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.442104    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442451 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.442134    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8ppk\" (UniqueName: \"kubernetes.io/projected/fd293127-52cd-46ea-aa6d-6c3d8af86732-kube-api-access-v8ppk\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.442451 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.442167    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.542954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.542913    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.542954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.542954    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.542970    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8ppk\" (UniqueName: \"kubernetes.io/projected/fd293127-52cd-46ea-aa6d-6c3d8af86732-kube-api-access-v8ppk\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.542992    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543016    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543039    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543162    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543439 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543230    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543439 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543267    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd293127-52cd-46ea-aa6d-6c3d8af86732-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543439 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543299    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543439 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543354    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543439 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543426    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543454    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-web-config\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543485    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd293127-52cd-46ea-aa6d-6c3d8af86732-config-out\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543508    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543529    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543554    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543589    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543614    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-config\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.543686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543636    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.544094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543721    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.544094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543797    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.544094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.543899    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.544692 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.544662    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.545604 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.545577    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.546269 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.546201    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.546805 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.546468    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.546805 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.546667    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.546805 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.546774    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.547070 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.546992    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.547287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.547251    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd293127-52cd-46ea-aa6d-6c3d8af86732-config-out\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.547823 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.547410    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.547823 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.547622    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd293127-52cd-46ea-aa6d-6c3d8af86732-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.547823 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.547772    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.548630 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.548602    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-config\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.548898 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.548845    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.549161 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.549143    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.549264 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.549238    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd293127-52cd-46ea-aa6d-6c3d8af86732-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.549536 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.549518    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd293127-52cd-46ea-aa6d-6c3d8af86732-web-config\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.552055 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.552035    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8ppk\" (UniqueName: \"kubernetes.io/projected/fd293127-52cd-46ea-aa6d-6c3d8af86732-kube-api-access-v8ppk\") pod \"prometheus-k8s-0\" (UID: \"fd293127-52cd-46ea-aa6d-6c3d8af86732\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.633805 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.633765    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:36.917994 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:36.917960    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 05 06:29:36.920375 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:29:36.920349    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd293127_52cd_46ea_aa6d_6c3d8af86732.slice/crio-210f1683c7e49855a4d0624402dd95c66a3cb0c6c16feb45577b6a8b74db2d36 WatchSource:0}: Error finding container 210f1683c7e49855a4d0624402dd95c66a3cb0c6c16feb45577b6a8b74db2d36: Status 404 returned error can't find the container with id 210f1683c7e49855a4d0624402dd95c66a3cb0c6c16feb45577b6a8b74db2d36
Sep 05 06:29:37.258346 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:37.258312    2573 generic.go:356] "Generic (PLEG): container finished" podID="fd293127-52cd-46ea-aa6d-6c3d8af86732" containerID="e75d87bf7fdd3fe76bbee8d17f8e17c10d8679a4ededf82cb1f4f89cc17a876c" exitCode=0
Sep 05 06:29:37.258774 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:37.258393    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd293127-52cd-46ea-aa6d-6c3d8af86732","Type":"ContainerDied","Data":"e75d87bf7fdd3fe76bbee8d17f8e17c10d8679a4ededf82cb1f4f89cc17a876c"}
Sep 05 06:29:37.258774 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:37.258505    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd293127-52cd-46ea-aa6d-6c3d8af86732","Type":"ContainerStarted","Data":"210f1683c7e49855a4d0624402dd95c66a3cb0c6c16feb45577b6a8b74db2d36"}
Sep 05 06:29:38.155669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:38.155623    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="437bcc3b-1b62-4905-bb17-09a1f31a89f8" path="/var/lib/kubelet/pods/437bcc3b-1b62-4905-bb17-09a1f31a89f8/volumes"
Sep 05 06:29:38.267068 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:38.267032    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd293127-52cd-46ea-aa6d-6c3d8af86732","Type":"ContainerStarted","Data":"7397b0d5ef84dee1ef10d8184cec1c1a4eb99b4c5cd319a0c4e280bf8c9f5101"}
Sep 05 06:29:38.267554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:38.267108    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd293127-52cd-46ea-aa6d-6c3d8af86732","Type":"ContainerStarted","Data":"fc2e28b20c21404c2e2e9a71e02be36481d8393c7dd39678631253745831dc63"}
Sep 05 06:29:38.267554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:38.267154    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd293127-52cd-46ea-aa6d-6c3d8af86732","Type":"ContainerStarted","Data":"0388c8f15a9802d9cb6321b8d6a777b845d1ebb6c0b0ee017282394eb6305d81"}
Sep 05 06:29:38.267554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:38.267195    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd293127-52cd-46ea-aa6d-6c3d8af86732","Type":"ContainerStarted","Data":"ada3941ef717f80c7963d35361608bcc8d145c960ca23af7ea96f53a868407f0"}
Sep 05 06:29:38.267554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:38.267244    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd293127-52cd-46ea-aa6d-6c3d8af86732","Type":"ContainerStarted","Data":"df66b02bbe522e4d6642e019745e32d7a3a0883ef328e83553caa1d03f348207"}
Sep 05 06:29:38.267554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:38.267281    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd293127-52cd-46ea-aa6d-6c3d8af86732","Type":"ContainerStarted","Data":"ac9194d80715786efbd120c7494f2c3412e9481ce37ebe34f5c55107304c3c3b"}
Sep 05 06:29:38.297790 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:38.297736    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.275471739 podStartE2EDuration="2.297717576s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.245837ms" isStatefulPod=false podCreationTimestamp="2026-09-05 06:29:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:29:38.295903366 +0000 UTC m=+226.663950579" watchObservedRunningTime="2026-09-05 06:29:38.297717576 +0000 UTC m=+226.665764784"
Sep 05 06:29:38.444829 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:38.444740    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:38.509117 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:38.509083    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:40.320386 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:40.320356    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:40.421991 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:40.421955    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:40.487125 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:40.487081    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:41.634550 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:41.634514    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:29:46.653323 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:46.653280    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:46.752989 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:46.752953    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:29:46.829230 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:29:46.829182    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:06.387792 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:06.387754    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:06.484179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:06.484136    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:18.850720 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:18.850630    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:18.910995 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:18.910947    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:27.839293 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:27.839254    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:29.250195 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:29.250156    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:31.468563 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:31.468534    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:31.575660 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:31.575623    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:33.523447 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:33.523409    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:33.621553 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:33.621515    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:33.779979 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:33.779895    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:33.879606 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:33.879572    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:36.196559 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:36.196521    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:36.394706 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:36.394678    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:36.634549 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:36.634507    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:30:36.649981 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:36.649956    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:30:37.508738 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:37.508711    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 05 06:30:38.417480 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:38.417436    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:38.644138 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:38.644104    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:38.744336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:38.744300    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:43.885811 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:43.885774    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:44.857824 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:44.857779    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:44.970086 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:44.970036    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:45.380294 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.380259    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nzn4x"]
Sep 05 06:30:45.383332 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.383316    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nzn4x"
Sep 05 06:30:45.385872 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.385836    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 05 06:30:45.386925 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.386894    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-g4p8l\""
Sep 05 06:30:45.386925 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.386894    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 05 06:30:45.394458 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.394435    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nzn4x"]
Sep 05 06:30:45.540460 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.540402    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfwnx\" (UniqueName: \"kubernetes.io/projected/a9f47253-4666-45ab-a35f-8f659fb1227e-kube-api-access-vfwnx\") pod \"cert-manager-55b4d954d5-nzn4x\" (UID: \"a9f47253-4666-45ab-a35f-8f659fb1227e\") " pod="cert-manager/cert-manager-55b4d954d5-nzn4x"
Sep 05 06:30:45.540645 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.540477    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9f47253-4666-45ab-a35f-8f659fb1227e-bound-sa-token\") pod \"cert-manager-55b4d954d5-nzn4x\" (UID: \"a9f47253-4666-45ab-a35f-8f659fb1227e\") " pod="cert-manager/cert-manager-55b4d954d5-nzn4x"
Sep 05 06:30:45.641954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.641837    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfwnx\" (UniqueName: \"kubernetes.io/projected/a9f47253-4666-45ab-a35f-8f659fb1227e-kube-api-access-vfwnx\") pod \"cert-manager-55b4d954d5-nzn4x\" (UID: \"a9f47253-4666-45ab-a35f-8f659fb1227e\") " pod="cert-manager/cert-manager-55b4d954d5-nzn4x"
Sep 05 06:30:45.641954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.641919    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9f47253-4666-45ab-a35f-8f659fb1227e-bound-sa-token\") pod \"cert-manager-55b4d954d5-nzn4x\" (UID: \"a9f47253-4666-45ab-a35f-8f659fb1227e\") " pod="cert-manager/cert-manager-55b4d954d5-nzn4x"
Sep 05 06:30:45.651483 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.651452    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9f47253-4666-45ab-a35f-8f659fb1227e-bound-sa-token\") pod \"cert-manager-55b4d954d5-nzn4x\" (UID: \"a9f47253-4666-45ab-a35f-8f659fb1227e\") " pod="cert-manager/cert-manager-55b4d954d5-nzn4x"
Sep 05 06:30:45.651607 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.651591    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfwnx\" (UniqueName: \"kubernetes.io/projected/a9f47253-4666-45ab-a35f-8f659fb1227e-kube-api-access-vfwnx\") pod \"cert-manager-55b4d954d5-nzn4x\" (UID: \"a9f47253-4666-45ab-a35f-8f659fb1227e\") " pod="cert-manager/cert-manager-55b4d954d5-nzn4x"
Sep 05 06:30:45.693329 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.693295    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nzn4x"
Sep 05 06:30:45.963577 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:45.963550    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nzn4x"]
Sep 05 06:30:45.965933 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:30:45.965910    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9f47253_4666_45ab_a35f_8f659fb1227e.slice/crio-d19179719535a45e3f924a10ca648953466b73e8a0221e9ed346658f2094b49f WatchSource:0}: Error finding container d19179719535a45e3f924a10ca648953466b73e8a0221e9ed346658f2094b49f: Status 404 returned error can't find the container with id d19179719535a45e3f924a10ca648953466b73e8a0221e9ed346658f2094b49f
Sep 05 06:30:46.527479 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:46.527447    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nzn4x" event={"ID":"a9f47253-4666-45ab-a35f-8f659fb1227e","Type":"ContainerStarted","Data":"d19179719535a45e3f924a10ca648953466b73e8a0221e9ed346658f2094b49f"}
Sep 05 06:30:49.543677 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:49.543645    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nzn4x" event={"ID":"a9f47253-4666-45ab-a35f-8f659fb1227e","Type":"ContainerStarted","Data":"7afa7259092dee216f552c5b2f04c9595928970be6e85531823d71d5abcd535c"}
Sep 05 06:30:49.562197 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:49.562150    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-nzn4x" podStartSLOduration=1.541044366 podStartE2EDuration="4.562133163s" totalImagesPullingTime="3.021088797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:30:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:30:49.560997783 +0000 UTC m=+297.929044992" watchObservedRunningTime="2026-09-05 06:30:49.562133163 +0000 UTC m=+297.930180373"
Sep 05 06:30:51.158496 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:51.158458    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:51.263952 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:51.263915    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:52.062677 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:52.062646    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:30:52.064609 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:52.064586    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:30:52.079840 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:52.079810    2573 kubelet.go:1648] "Image garbage collection succeeded"
Sep 05 06:30:53.556974 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:53.556938    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:56.354469 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:56.354429    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:56.427864 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:56.427825    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:56.543972 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:56.543936    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:30:56.620104 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:30:56.620018    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:07.243874 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:07.243819    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:07.344270 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:07.344226    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:07.888879 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:07.888816    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:08.142316 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.142223    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:08.243691 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.243655    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:08.287656 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.287623    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"]
Sep 05 06:31:08.294305 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.294287    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.296862 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.296827    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 05 06:31:08.296993 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.296861    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2f8p8\""
Sep 05 06:31:08.296993 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.296879    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 05 06:31:08.296993 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.296883    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 05 06:31:08.296993 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.296954    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 05 06:31:08.304955 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.304932    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"]
Sep 05 06:31:08.330940 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.330909    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55738568-3181-4961-a96a-6153a9456780-tmp\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.330940 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.330942    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55738568-3181-4961-a96a-6153a9456780-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.331154 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.330962    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55738568-3181-4961-a96a-6153a9456780-webhook-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.331154 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.330982    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55738568-3181-4961-a96a-6153a9456780-manifests\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.331154 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.331063    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnqr5\" (UniqueName: \"kubernetes.io/projected/55738568-3181-4961-a96a-6153a9456780-kube-api-access-qnqr5\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.432052 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.431962    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55738568-3181-4961-a96a-6153a9456780-tmp\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.432052 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.431999    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55738568-3181-4961-a96a-6153a9456780-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.432052 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.432018    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55738568-3181-4961-a96a-6153a9456780-webhook-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.432334 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.432154    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55738568-3181-4961-a96a-6153a9456780-manifests\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.432334 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.432209    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnqr5\" (UniqueName: \"kubernetes.io/projected/55738568-3181-4961-a96a-6153a9456780-kube-api-access-qnqr5\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.432612 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.432587    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55738568-3181-4961-a96a-6153a9456780-manifests\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.434388 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.434361    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55738568-3181-4961-a96a-6153a9456780-tmp\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.434603 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.434586    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55738568-3181-4961-a96a-6153a9456780-webhook-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.435169 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.435151    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55738568-3181-4961-a96a-6153a9456780-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.443818 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.443795    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnqr5\" (UniqueName: \"kubernetes.io/projected/55738568-3181-4961-a96a-6153a9456780-kube-api-access-qnqr5\") pod \"opendatahub-operator-controller-manager-555dc54ddd-nmjt2\" (UID: \"55738568-3181-4961-a96a-6153a9456780\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.450932 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.450903    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:08.551927 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.551894    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:08.604801 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.604765    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:08.657278 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.657168    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:08.755238 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.755211    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:08.859918 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.859875    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:08.902902 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.902737    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"]
Sep 05 06:31:08.905899 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:31:08.905873    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55738568_3181_4961_a96a_6153a9456780.slice/crio-0e62724ba8ce6a0e4cf101062c3d1fb2d687cc7634e47f983848543305ea0dd2 WatchSource:0}: Error finding container 0e62724ba8ce6a0e4cf101062c3d1fb2d687cc7634e47f983848543305ea0dd2: Status 404 returned error can't find the container with id 0e62724ba8ce6a0e4cf101062c3d1fb2d687cc7634e47f983848543305ea0dd2
Sep 05 06:31:08.907661 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.907642    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 06:31:08.965267 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:08.965231    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:09.628596 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:09.628562    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2" event={"ID":"55738568-3181-4961-a96a-6153a9456780","Type":"ContainerStarted","Data":"0e62724ba8ce6a0e4cf101062c3d1fb2d687cc7634e47f983848543305ea0dd2"}
Sep 05 06:31:12.650938 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:12.650892    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2" event={"ID":"55738568-3181-4961-a96a-6153a9456780","Type":"ContainerStarted","Data":"da1d2800d356d5d76c0569fbd8781799ab76a0e2cf96c743384258d1b50e4fb8"}
Sep 05 06:31:13.657464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:13.657432    2573 generic.go:356] "Generic (PLEG): container finished" podID="55738568-3181-4961-a96a-6153a9456780" containerID="da1d2800d356d5d76c0569fbd8781799ab76a0e2cf96c743384258d1b50e4fb8" exitCode=0
Sep 05 06:31:13.657842 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:13.657480    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2" event={"ID":"55738568-3181-4961-a96a-6153a9456780","Type":"ContainerDied","Data":"da1d2800d356d5d76c0569fbd8781799ab76a0e2cf96c743384258d1b50e4fb8"}
Sep 05 06:31:14.668504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:14.668461    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2" event={"ID":"55738568-3181-4961-a96a-6153a9456780","Type":"ContainerStarted","Data":"5220e7216096d11e5833ce744cda2acffb27b79299c3e7913a6832e9dd851c3f"}
Sep 05 06:31:14.668956 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:14.668553    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:14.707449 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:14.707381    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2" podStartSLOduration=2.402120125 podStartE2EDuration="6.707361307s" totalImagesPullingTime="3.273406005s" totalInitContainerRuntime="1.031835177s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:31:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:31:14.706010909 +0000 UTC m=+323.074058119" watchObservedRunningTime="2026-09-05 06:31:14.707361307 +0000 UTC m=+323.075408517"
Sep 05 06:31:15.746381 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:15.746334    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:15.850910 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:15.850875    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:19.659776 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:19.659740    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:19.760913 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:19.760882    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:25.674491 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:25.674453    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-nmjt2"
Sep 05 06:31:26.355367 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.355321    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"]
Sep 05 06:31:26.362783 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.362757    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.365356 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.365331    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 05 06:31:26.366272 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.366249    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 05 06:31:26.366399 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.366277    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 05 06:31:26.366399 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.366286    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 05 06:31:26.366399 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.366254    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 05 06:31:26.366399 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.366308    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tslgz\""
Sep 05 06:31:26.372714 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.372692    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"]
Sep 05 06:31:26.494705 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.494662    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-manager-config\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.494908 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.494748    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-cert\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.494908 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.494815    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-metrics-cert\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.494908 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.494900    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnmb7\" (UniqueName: \"kubernetes.io/projected/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-kube-api-access-pnmb7\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.595540 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.595499    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-cert\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.595540 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.595550    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-metrics-cert\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.595745 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.595587    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnmb7\" (UniqueName: \"kubernetes.io/projected/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-kube-api-access-pnmb7\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.595745 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.595611    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-manager-config\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.596329 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.596306    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-manager-config\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.598099 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.598072    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-metrics-cert\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.598213 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.598124    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-cert\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.612874 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.612785    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnmb7\" (UniqueName: \"kubernetes.io/projected/35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0-kube-api-access-pnmb7\") pod \"lws-controller-manager-66f494d4f4-hlr98\" (UID: \"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0\") " pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.672963 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.672927    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:26.944478 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:26.944450    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"]
Sep 05 06:31:26.947276 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:31:26.947251    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c3ef1a_ae1e_4fcd_a407_e60cb4c67fa0.slice/crio-6470f1bb346a7f8dc843af2fbe1eeba51732e2041ca03c8c7dc53e7c795a27a7 WatchSource:0}: Error finding container 6470f1bb346a7f8dc843af2fbe1eeba51732e2041ca03c8c7dc53e7c795a27a7: Status 404 returned error can't find the container with id 6470f1bb346a7f8dc843af2fbe1eeba51732e2041ca03c8c7dc53e7c795a27a7
Sep 05 06:31:27.330553 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.330518    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:27.458860 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.458828    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:27.727981 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.727819    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98" event={"ID":"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0","Type":"ContainerStarted","Data":"6470f1bb346a7f8dc843af2fbe1eeba51732e2041ca03c8c7dc53e7c795a27a7"}
Sep 05 06:31:27.781447 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.781355    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:27.885749 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.884642    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"]
Sep 05 06:31:27.888622 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.888594    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:27.894318 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.893826    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 05 06:31:27.897594 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.897212    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 05 06:31:27.897594 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.897258    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 05 06:31:27.897594 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.897215    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 05 06:31:27.897594 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.897464    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sxfnq\""
Sep 05 06:31:27.901692 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:27.901549    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"]
Sep 05 06:31:28.009044 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.009005    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpklf\" (UniqueName: \"kubernetes.io/projected/806212c4-b028-46c6-acc8-0cd1c250c7a8-kube-api-access-qpklf\") pod \"kube-auth-proxy-6bd96764bd-8zzgm\" (UID: \"806212c4-b028-46c6-acc8-0cd1c250c7a8\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:28.009529 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.009111    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/806212c4-b028-46c6-acc8-0cd1c250c7a8-tls-certs\") pod \"kube-auth-proxy-6bd96764bd-8zzgm\" (UID: \"806212c4-b028-46c6-acc8-0cd1c250c7a8\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:28.009529 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.009245    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/806212c4-b028-46c6-acc8-0cd1c250c7a8-tmp\") pod \"kube-auth-proxy-6bd96764bd-8zzgm\" (UID: \"806212c4-b028-46c6-acc8-0cd1c250c7a8\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:28.110500 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.110455    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/806212c4-b028-46c6-acc8-0cd1c250c7a8-tmp\") pod \"kube-auth-proxy-6bd96764bd-8zzgm\" (UID: \"806212c4-b028-46c6-acc8-0cd1c250c7a8\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:28.110712 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.110547    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpklf\" (UniqueName: \"kubernetes.io/projected/806212c4-b028-46c6-acc8-0cd1c250c7a8-kube-api-access-qpklf\") pod \"kube-auth-proxy-6bd96764bd-8zzgm\" (UID: \"806212c4-b028-46c6-acc8-0cd1c250c7a8\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:28.110712 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.110580    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/806212c4-b028-46c6-acc8-0cd1c250c7a8-tls-certs\") pod \"kube-auth-proxy-6bd96764bd-8zzgm\" (UID: \"806212c4-b028-46c6-acc8-0cd1c250c7a8\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:28.110834 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:31:28.110752    2573 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 05 06:31:28.110834 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:31:28.110817    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/806212c4-b028-46c6-acc8-0cd1c250c7a8-tls-certs podName:806212c4-b028-46c6-acc8-0cd1c250c7a8 nodeName:}" failed. No retries permitted until 2026-09-05 06:31:28.610796295 +0000 UTC m=+336.978843487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/806212c4-b028-46c6-acc8-0cd1c250c7a8-tls-certs") pod "kube-auth-proxy-6bd96764bd-8zzgm" (UID: "806212c4-b028-46c6-acc8-0cd1c250c7a8") : secret "kube-auth-proxy-tls" not found
Sep 05 06:31:28.113893 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.113866    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/806212c4-b028-46c6-acc8-0cd1c250c7a8-tmp\") pod \"kube-auth-proxy-6bd96764bd-8zzgm\" (UID: \"806212c4-b028-46c6-acc8-0cd1c250c7a8\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:28.121903 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.121823    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpklf\" (UniqueName: \"kubernetes.io/projected/806212c4-b028-46c6-acc8-0cd1c250c7a8-kube-api-access-qpklf\") pod \"kube-auth-proxy-6bd96764bd-8zzgm\" (UID: \"806212c4-b028-46c6-acc8-0cd1c250c7a8\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:28.329145 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.329059    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:28.616783 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.616697    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/806212c4-b028-46c6-acc8-0cd1c250c7a8-tls-certs\") pod \"kube-auth-proxy-6bd96764bd-8zzgm\" (UID: \"806212c4-b028-46c6-acc8-0cd1c250c7a8\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:28.619305 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.619274    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/806212c4-b028-46c6-acc8-0cd1c250c7a8-tls-certs\") pod \"kube-auth-proxy-6bd96764bd-8zzgm\" (UID: \"806212c4-b028-46c6-acc8-0cd1c250c7a8\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:28.802805 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:28.802768    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"
Sep 05 06:31:29.442473 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:29.442430    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:29.491743 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:29.491711    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm"]
Sep 05 06:31:29.493923 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:31:29.493893    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod806212c4_b028_46c6_acc8_0cd1c250c7a8.slice/crio-a8e14203401af3c55c7307807385581eb4af3fdd34b88516ff3f43145395650c WatchSource:0}: Error finding container a8e14203401af3c55c7307807385581eb4af3fdd34b88516ff3f43145395650c: Status 404 returned error can't find the container with id a8e14203401af3c55c7307807385581eb4af3fdd34b88516ff3f43145395650c
Sep 05 06:31:29.530019 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:29.529986    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:29.738012 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:29.737974    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98" event={"ID":"35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0","Type":"ContainerStarted","Data":"d5eef77a51814bb7e1aaed31a37051469e002f89e366a2f7eaffa2968f01b657"}
Sep 05 06:31:29.738178 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:29.738068    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:29.739283 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:29.739249    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm" event={"ID":"806212c4-b028-46c6-acc8-0cd1c250c7a8","Type":"ContainerStarted","Data":"a8e14203401af3c55c7307807385581eb4af3fdd34b88516ff3f43145395650c"}
Sep 05 06:31:29.761083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:29.760993    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98" podStartSLOduration=1.430423109 podStartE2EDuration="3.760975263s" totalImagesPullingTime="2.330552154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:31:29.760006055 +0000 UTC m=+338.128053275" watchObservedRunningTime="2026-09-05 06:31:29.760975263 +0000 UTC m=+338.129022474"
Sep 05 06:31:30.630805 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:30.630763    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:32.756493 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:32.756456    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm" event={"ID":"806212c4-b028-46c6-acc8-0cd1c250c7a8","Type":"ContainerStarted","Data":"4df1356cceb79bf191917934d701b5f84ea66491cb8fd9195bec443538faf2b2"}
Sep 05 06:31:32.776795 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:32.776743    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bd96764bd-8zzgm" podStartSLOduration=3.352625692 podStartE2EDuration="5.776726054s" totalImagesPullingTime="2.424100362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:31:32.77547515 +0000 UTC m=+341.143522361" watchObservedRunningTime="2026-09-05 06:31:32.776726054 +0000 UTC m=+341.144773264"
Sep 05 06:31:34.558087 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:34.558048    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:34.659364 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:34.659329    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:35.239088 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:35.239052    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:35.382963 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:35.382926    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:35.524215 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:35.524176    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:38.723394 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:38.723360    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:38.814133 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:38.814090    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:38.914187 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:38.914149    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:39.014023 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:39.013990    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:39.329682 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:39.329598    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:40.283258 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:40.283221    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:40.748524 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:40.748493    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66f494d4f4-hlr98"
Sep 05 06:31:49.224731 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:49.224644    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:49.360119 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:49.360079    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:49.456866 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:49.456834    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:49.616973 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:49.616925    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:50.577509 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:50.577478    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:50.680960 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:50.680921    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:31:50.778125 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:31:50.778091    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:32:39.717617 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:32:39.717583    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:32:40.813934 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:32:40.813899    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:32:52.637481 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:32:52.637447    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:32:55.470936 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:32:55.470893    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:32:57.223906 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:32:57.223873    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:32:57.640180 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:32:57.640136    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:32:57.877832 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:32:57.877800    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:32:58.046611 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:32:58.046581    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:32:59.093324 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:32:59.093276    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:00.230061 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:00.230025    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:01.440291 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:01.440256    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:01.682896 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:01.682833    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:01.839432 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:01.839392    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:01.940072 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:01.940026    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:05.258744 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:05.258709    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:05.362613 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:05.362579    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:08.230592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:08.230559    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:08.330809 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:08.330769    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:09.285390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.285351    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:09.285795 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.285646    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"]
Sep 05 06:33:09.288970 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.288956    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:09.291506 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.291479    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 05 06:33:09.291740 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.291725    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 05 06:33:09.292637 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.292594    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 05 06:33:09.292637 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.292615    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 05 06:33:09.292637 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.292624    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qckkf\""
Sep 05 06:33:09.295254 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.295233    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"]
Sep 05 06:33:09.388019 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.387985    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-d8qwd\" (UID: \"9e9a0e3c-9095-4760-b88a-b9b364bba2cc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:09.388019 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.388022    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjg7v\" (UniqueName: \"kubernetes.io/projected/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-kube-api-access-pjg7v\") pod \"kuadrant-console-plugin-598f86468b-d8qwd\" (UID: \"9e9a0e3c-9095-4760-b88a-b9b364bba2cc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:09.388212 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.388056    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-d8qwd\" (UID: \"9e9a0e3c-9095-4760-b88a-b9b364bba2cc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:09.489032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.489004    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-d8qwd\" (UID: \"9e9a0e3c-9095-4760-b88a-b9b364bba2cc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:09.489199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.489037    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjg7v\" (UniqueName: \"kubernetes.io/projected/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-kube-api-access-pjg7v\") pod \"kuadrant-console-plugin-598f86468b-d8qwd\" (UID: \"9e9a0e3c-9095-4760-b88a-b9b364bba2cc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:09.489199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.489064    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-d8qwd\" (UID: \"9e9a0e3c-9095-4760-b88a-b9b364bba2cc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:09.489199 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:33:09.489172    2573 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 05 06:33:09.489321 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:33:09.489243    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-plugin-serving-cert podName:9e9a0e3c-9095-4760-b88a-b9b364bba2cc nodeName:}" failed. No retries permitted until 2026-09-05 06:33:09.98922594 +0000 UTC m=+438.357273148 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-plugin-serving-cert") pod "kuadrant-console-plugin-598f86468b-d8qwd" (UID: "9e9a0e3c-9095-4760-b88a-b9b364bba2cc") : secret "plugin-serving-cert" not found
Sep 05 06:33:09.489686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.489666    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-d8qwd\" (UID: \"9e9a0e3c-9095-4760-b88a-b9b364bba2cc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:09.500323 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.500296    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjg7v\" (UniqueName: \"kubernetes.io/projected/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-kube-api-access-pjg7v\") pod \"kuadrant-console-plugin-598f86468b-d8qwd\" (UID: \"9e9a0e3c-9095-4760-b88a-b9b364bba2cc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:09.600901 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.600797    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:09.994891 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.994841    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-d8qwd\" (UID: \"9e9a0e3c-9095-4760-b88a-b9b364bba2cc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:09.997338 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:09.997315    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e9a0e3c-9095-4760-b88a-b9b364bba2cc-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-d8qwd\" (UID: \"9e9a0e3c-9095-4760-b88a-b9b364bba2cc\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:10.199125 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:10.199095    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"
Sep 05 06:33:10.468915 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:10.468894    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd"]
Sep 05 06:33:10.472974 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:33:10.471899    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e9a0e3c_9095_4760_b88a_b9b364bba2cc.slice/crio-ff14c4fbc397ab627db2bd0880a257e10c19855ca89828b269413086d661f676 WatchSource:0}: Error finding container ff14c4fbc397ab627db2bd0880a257e10c19855ca89828b269413086d661f676: Status 404 returned error can't find the container with id ff14c4fbc397ab627db2bd0880a257e10c19855ca89828b269413086d661f676
Sep 05 06:33:11.194310 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:11.194270    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd" event={"ID":"9e9a0e3c-9095-4760-b88a-b9b364bba2cc","Type":"ContainerStarted","Data":"ff14c4fbc397ab627db2bd0880a257e10c19855ca89828b269413086d661f676"}
Sep 05 06:33:12.342070 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:12.342030    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:12.461078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:12.461042    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:13.228441 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:13.228405    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:13.326336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:13.326295    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:13.425644 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:13.425605    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:15.215706 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:15.215671    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd" event={"ID":"9e9a0e3c-9095-4760-b88a-b9b364bba2cc","Type":"ContainerStarted","Data":"857ba0cab7aa8371c6ab17b143664845589c85f37d8b3a14b916a4ff64a60ff9"}
Sep 05 06:33:15.234245 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:15.234194    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-d8qwd" podStartSLOduration=1.992518521 podStartE2EDuration="6.23417822s" totalImagesPullingTime="4.241659699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:15.231999921 +0000 UTC m=+443.600047135" watchObservedRunningTime="2026-09-05 06:33:15.23417822 +0000 UTC m=+443.602225443"
Sep 05 06:33:19.395514 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.395476    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"]
Sep 05 06:33:19.398929 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.398910    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:19.402442 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.402419    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gkp9h\""
Sep 05 06:33:19.417164 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.417136    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"]
Sep 05 06:33:19.581353 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.581312    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5qqcm\" (UID: \"e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:19.581538 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.581409    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs2h8\" (UniqueName: \"kubernetes.io/projected/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-kube-api-access-bs2h8\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5qqcm\" (UID: \"e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:19.682442 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.682360    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs2h8\" (UniqueName: \"kubernetes.io/projected/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-kube-api-access-bs2h8\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5qqcm\" (UID: \"e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:19.682568 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.682440    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5qqcm\" (UID: \"e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:19.682846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.682827    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5qqcm\" (UID: \"e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:19.692975 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.692941    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs2h8\" (UniqueName: \"kubernetes.io/projected/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-kube-api-access-bs2h8\") pod \"kuadrant-operator-controller-manager-6bd956ccf-5qqcm\" (UID: \"e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:19.708748 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.708722    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:19.988841 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:19.988790    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"]
Sep 05 06:33:20.237930 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.237892    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm" event={"ID":"e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e","Type":"ContainerStarted","Data":"39ceeca16c582dd74c9e7c290d64badf108ae19ceb248e987999adac6beb398a"}
Sep 05 06:33:20.310234 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.310198    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"]
Sep 05 06:33:20.327686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.327648    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"]
Sep 05 06:33:20.347573 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.347532    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"]
Sep 05 06:33:20.352136 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.352107    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:20.375528 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.375491    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"]
Sep 05 06:33:20.489606 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.489563    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adc94ff0-0bcd-4268-8a00-0be235df91c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-nz2jx\" (UID: \"adc94ff0-0bcd-4268-8a00-0be235df91c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:20.489978 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.489619    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf4ch\" (UniqueName: \"kubernetes.io/projected/adc94ff0-0bcd-4268-8a00-0be235df91c8-kube-api-access-zf4ch\") pod \"kuadrant-operator-controller-manager-6bd956ccf-nz2jx\" (UID: \"adc94ff0-0bcd-4268-8a00-0be235df91c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:20.591140 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.591050    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adc94ff0-0bcd-4268-8a00-0be235df91c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-nz2jx\" (UID: \"adc94ff0-0bcd-4268-8a00-0be235df91c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:20.591140 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.591114    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf4ch\" (UniqueName: \"kubernetes.io/projected/adc94ff0-0bcd-4268-8a00-0be235df91c8-kube-api-access-zf4ch\") pod \"kuadrant-operator-controller-manager-6bd956ccf-nz2jx\" (UID: \"adc94ff0-0bcd-4268-8a00-0be235df91c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:20.591463 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.591445    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adc94ff0-0bcd-4268-8a00-0be235df91c8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-nz2jx\" (UID: \"adc94ff0-0bcd-4268-8a00-0be235df91c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:20.602317 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.602287    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf4ch\" (UniqueName: \"kubernetes.io/projected/adc94ff0-0bcd-4268-8a00-0be235df91c8-kube-api-access-zf4ch\") pod \"kuadrant-operator-controller-manager-6bd956ccf-nz2jx\" (UID: \"adc94ff0-0bcd-4268-8a00-0be235df91c8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:20.663046 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.663012    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:20.963296 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:20.962921    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"]
Sep 05 06:33:20.966997 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:33:20.966961    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadc94ff0_0bcd_4268_8a00_0be235df91c8.slice/crio-be02c6bfc7ffb9cadbb65de5dd7e36be5709b308b83cb7aa83a0d89d1c508cb8 WatchSource:0}: Error finding container be02c6bfc7ffb9cadbb65de5dd7e36be5709b308b83cb7aa83a0d89d1c508cb8: Status 404 returned error can't find the container with id be02c6bfc7ffb9cadbb65de5dd7e36be5709b308b83cb7aa83a0d89d1c508cb8
Sep 05 06:33:21.245457 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:21.245424    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx" event={"ID":"adc94ff0-0bcd-4268-8a00-0be235df91c8","Type":"ContainerStarted","Data":"be02c6bfc7ffb9cadbb65de5dd7e36be5709b308b83cb7aa83a0d89d1c508cb8"}
Sep 05 06:33:22.058793 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:22.058714    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:22.161949 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:22.161833    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:24.637715 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:24.637626    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:24.738413 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:24.738356    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:25.268776 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.268741    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm" podUID="e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e" containerName="manager" containerID="cri-o://f882062d5e05f04b2f2549a422270ccd120240278335d63a9e1a97e440d0aeae" gracePeriod=2
Sep 05 06:33:25.270663 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.270634    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx" event={"ID":"adc94ff0-0bcd-4268-8a00-0be235df91c8","Type":"ContainerStarted","Data":"c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9"}
Sep 05 06:33:25.270955 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.270929    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:25.361224 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.361186    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:25.461437 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.461410    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:25.511346 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.511325    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:25.514150 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:33:25.514123    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-5qqcm\" is forbidden: User \"system:node:ip-10-0-128-174.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-174.ec2.internal' and this object" podUID="e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:25.538470 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.538404    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-extensions-socket-volume\") pod \"e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e\" (UID: \"e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e\") "
Sep 05 06:33:25.538470 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.538437    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs2h8\" (UniqueName: \"kubernetes.io/projected/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-kube-api-access-bs2h8\") pod \"e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e\" (UID: \"e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e\") "
Sep 05 06:33:25.538703 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.538679    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e" (UID: "e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:33:25.540634 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.540605    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-kube-api-access-bs2h8" (OuterVolumeSpecName: "kube-api-access-bs2h8") pod "e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e" (UID: "e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e"). InnerVolumeSpecName "kube-api-access-bs2h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:33:25.640000 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.639960    2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-extensions-socket-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:33:25.640000 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:25.639994    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs2h8\" (UniqueName: \"kubernetes.io/projected/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e-kube-api-access-bs2h8\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:33:26.155318 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:26.155283    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e" path="/var/lib/kubelet/pods/e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e/volumes"
Sep 05 06:33:26.277641 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:26.277610    2573 generic.go:356] "Generic (PLEG): container finished" podID="e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e" containerID="f882062d5e05f04b2f2549a422270ccd120240278335d63a9e1a97e440d0aeae" exitCode=2
Sep 05 06:33:26.277807 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:26.277670    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:26.277807 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:26.277699    2573 scope.go:117] "RemoveContainer" containerID="f882062d5e05f04b2f2549a422270ccd120240278335d63a9e1a97e440d0aeae"
Sep 05 06:33:26.282382 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:33:26.282349    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-5qqcm\" is forbidden: User \"system:node:ip-10-0-128-174.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-174.ec2.internal' and this object" podUID="e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-5qqcm"
Sep 05 06:33:26.285885 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:26.285869    2573 scope.go:117] "RemoveContainer" containerID="f882062d5e05f04b2f2549a422270ccd120240278335d63a9e1a97e440d0aeae"
Sep 05 06:33:26.286156 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:33:26.286138    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f882062d5e05f04b2f2549a422270ccd120240278335d63a9e1a97e440d0aeae\": container with ID starting with f882062d5e05f04b2f2549a422270ccd120240278335d63a9e1a97e440d0aeae not found: ID does not exist" containerID="f882062d5e05f04b2f2549a422270ccd120240278335d63a9e1a97e440d0aeae"
Sep 05 06:33:26.286216 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:26.286164    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f882062d5e05f04b2f2549a422270ccd120240278335d63a9e1a97e440d0aeae"} err="failed to get container status \"f882062d5e05f04b2f2549a422270ccd120240278335d63a9e1a97e440d0aeae\": rpc error: code = NotFound desc = could not find container \"f882062d5e05f04b2f2549a422270ccd120240278335d63a9e1a97e440d0aeae\": container with ID starting with f882062d5e05f04b2f2549a422270ccd120240278335d63a9e1a97e440d0aeae not found: ID does not exist"
Sep 05 06:33:27.631362 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:27.631322    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:27.732113 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:27.732077    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:33.229811 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:33.229764    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:33.331737 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:33.331708    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:34.047836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:34.047799    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:34.149179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:34.149147    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:34.249447 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:34.249413    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:34.349472 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:34.349385    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:36.200537 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:36.200501    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:36.280376 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:36.280345    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:36.300024 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:36.299991    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:36.308233 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:36.308180    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx" podStartSLOduration=12.946709079 podStartE2EDuration="16.30816462s" totalImagesPullingTime="3.361455541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:25.307549647 +0000 UTC m=+453.675596857" watchObservedRunningTime="2026-09-05 06:33:36.30816462 +0000 UTC m=+464.676211829"
Sep 05 06:33:36.380287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:36.380250    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:41.236074 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:41.236034    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:41.334581 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:41.334546    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:49.125391 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.125353    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"]
Sep 05 06:33:49.125898 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.125592    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx" podUID="adc94ff0-0bcd-4268-8a00-0be235df91c8" containerName="manager" containerID="cri-o://c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9" gracePeriod=10
Sep 05 06:33:49.362541 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.362518    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:49.384620 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.384569    2573 generic.go:356] "Generic (PLEG): container finished" podID="adc94ff0-0bcd-4268-8a00-0be235df91c8" containerID="c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9" exitCode=0
Sep 05 06:33:49.384620 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.384607    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"
Sep 05 06:33:49.384775 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.384640    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx" event={"ID":"adc94ff0-0bcd-4268-8a00-0be235df91c8","Type":"ContainerDied","Data":"c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9"}
Sep 05 06:33:49.384775 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.384705    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx" event={"ID":"adc94ff0-0bcd-4268-8a00-0be235df91c8","Type":"ContainerDied","Data":"be02c6bfc7ffb9cadbb65de5dd7e36be5709b308b83cb7aa83a0d89d1c508cb8"}
Sep 05 06:33:49.384775 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.384728    2573 scope.go:117] "RemoveContainer" containerID="c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9"
Sep 05 06:33:49.393562 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.393541    2573 scope.go:117] "RemoveContainer" containerID="c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9"
Sep 05 06:33:49.393921 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:33:49.393896    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9\": container with ID starting with c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9 not found: ID does not exist" containerID="c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9"
Sep 05 06:33:49.393999 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.393932    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9"} err="failed to get container status \"c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9\": rpc error: code = NotFound desc = could not find container \"c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9\": container with ID starting with c5e467ec8c5d65dafca976fae54fa2a8ac5bd80b290beed4d9324d200eff86c9 not found: ID does not exist"
Sep 05 06:33:49.444885 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.444840    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zf4ch\" (UniqueName: \"kubernetes.io/projected/adc94ff0-0bcd-4268-8a00-0be235df91c8-kube-api-access-zf4ch\") pod \"adc94ff0-0bcd-4268-8a00-0be235df91c8\" (UID: \"adc94ff0-0bcd-4268-8a00-0be235df91c8\") "
Sep 05 06:33:49.445060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.444937    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adc94ff0-0bcd-4268-8a00-0be235df91c8-extensions-socket-volume\") pod \"adc94ff0-0bcd-4268-8a00-0be235df91c8\" (UID: \"adc94ff0-0bcd-4268-8a00-0be235df91c8\") "
Sep 05 06:33:49.445377 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.445354    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adc94ff0-0bcd-4268-8a00-0be235df91c8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "adc94ff0-0bcd-4268-8a00-0be235df91c8" (UID: "adc94ff0-0bcd-4268-8a00-0be235df91c8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:33:49.446794 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.446775    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adc94ff0-0bcd-4268-8a00-0be235df91c8-kube-api-access-zf4ch" (OuterVolumeSpecName: "kube-api-access-zf4ch") pod "adc94ff0-0bcd-4268-8a00-0be235df91c8" (UID: "adc94ff0-0bcd-4268-8a00-0be235df91c8"). InnerVolumeSpecName "kube-api-access-zf4ch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:33:49.546511 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.546469    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf4ch\" (UniqueName: \"kubernetes.io/projected/adc94ff0-0bcd-4268-8a00-0be235df91c8-kube-api-access-zf4ch\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:33:49.546511 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.546507    2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adc94ff0-0bcd-4268-8a00-0be235df91c8-extensions-socket-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:33:49.715950 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.715917    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"]
Sep 05 06:33:49.723995 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:49.723972    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-nz2jx"]
Sep 05 06:33:50.157787 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:50.157756    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adc94ff0-0bcd-4268-8a00-0be235df91c8" path="/var/lib/kubelet/pods/adc94ff0-0bcd-4268-8a00-0be235df91c8/volumes"
Sep 05 06:33:52.787246 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:52.787202    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:33:53.144468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:33:53.144377    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:06.722713 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.722677    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:06.725912 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.725885    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"]
Sep 05 06:34:06.726216 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.726204    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adc94ff0-0bcd-4268-8a00-0be235df91c8" containerName="manager"
Sep 05 06:34:06.726259 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.726218    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="adc94ff0-0bcd-4268-8a00-0be235df91c8" containerName="manager"
Sep 05 06:34:06.726259 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.726238    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e" containerName="manager"
Sep 05 06:34:06.726259 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.726244    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e" containerName="manager"
Sep 05 06:34:06.726356 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.726300    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4f9dc62-6f56-4b27-8a6d-2c15ba64a71e" containerName="manager"
Sep 05 06:34:06.726356 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.726308    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="adc94ff0-0bcd-4268-8a00-0be235df91c8" containerName="manager"
Sep 05 06:34:06.730628 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.730613    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:06.733589 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.733564    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 05 06:34:06.743252 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.743227    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"]
Sep 05 06:34:06.799715 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.799684    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxb9w\" (UniqueName: \"kubernetes.io/projected/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-kube-api-access-jxb9w\") pod \"limitador-limitador-859c4d96d6-mxj8r\" (UID: \"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:06.799904 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.799734    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-config-file\") pod \"limitador-limitador-859c4d96d6-mxj8r\" (UID: \"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:06.820332 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.820299    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"]
Sep 05 06:34:06.824390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.824362    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:06.900443 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.900408    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxb9w\" (UniqueName: \"kubernetes.io/projected/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-kube-api-access-jxb9w\") pod \"limitador-limitador-859c4d96d6-mxj8r\" (UID: \"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:06.900443 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.900454    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-config-file\") pod \"limitador-limitador-859c4d96d6-mxj8r\" (UID: \"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:06.901105 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.901087    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-config-file\") pod \"limitador-limitador-859c4d96d6-mxj8r\" (UID: \"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:06.909967 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.909941    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxb9w\" (UniqueName: \"kubernetes.io/projected/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-kube-api-access-jxb9w\") pod \"limitador-limitador-859c4d96d6-mxj8r\" (UID: \"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:06.925237 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:06.925202    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:07.040807 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:07.040775    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:07.336321 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:07.336221    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"]
Sep 05 06:34:07.341409 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:34:07.341378    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b4afbd_39e3_4e51_a2ed_d5f74ce5502a.slice/crio-85409dfafc8ad29e8cc33075cfb8c7353ed60b2dc1f1640b91f7be11490ab167 WatchSource:0}: Error finding container 85409dfafc8ad29e8cc33075cfb8c7353ed60b2dc1f1640b91f7be11490ab167: Status 404 returned error can't find the container with id 85409dfafc8ad29e8cc33075cfb8c7353ed60b2dc1f1640b91f7be11490ab167
Sep 05 06:34:07.472272 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:07.472237    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r" event={"ID":"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a","Type":"ContainerStarted","Data":"85409dfafc8ad29e8cc33075cfb8c7353ed60b2dc1f1640b91f7be11490ab167"}
Sep 05 06:34:07.621222 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:07.621112    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:07.720577 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:07.720545    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:09.483684 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:09.483650    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r" event={"ID":"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a","Type":"ContainerStarted","Data":"29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0"}
Sep 05 06:34:09.484181 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:09.483790    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:09.503754 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:09.503705    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r" podStartSLOduration=2.127096129 podStartE2EDuration="3.503690097s" totalImagesPullingTime="1.376593968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:34:09.501821388 +0000 UTC m=+497.869868599" watchObservedRunningTime="2026-09-05 06:34:09.503690097 +0000 UTC m=+497.871737307"
Sep 05 06:34:20.294492 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:20.294454    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"]
Sep 05 06:34:20.295051 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:20.294665    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r" podUID="45b4afbd-39e3-4e51-a2ed-d5f74ce5502a" containerName="limitador" containerID="cri-o://29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0" gracePeriod=30
Sep 05 06:34:20.295398 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:20.295315    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:21.237908 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.237883    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:21.324454 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.324415    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-config-file\") pod \"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a\" (UID: \"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a\") "
Sep 05 06:34:21.324839 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.324504    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxb9w\" (UniqueName: \"kubernetes.io/projected/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-kube-api-access-jxb9w\") pod \"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a\" (UID: \"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a\") "
Sep 05 06:34:21.324920 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.324873    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-config-file" (OuterVolumeSpecName: "config-file") pod "45b4afbd-39e3-4e51-a2ed-d5f74ce5502a" (UID: "45b4afbd-39e3-4e51-a2ed-d5f74ce5502a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:34:21.326569 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.326548    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-kube-api-access-jxb9w" (OuterVolumeSpecName: "kube-api-access-jxb9w") pod "45b4afbd-39e3-4e51-a2ed-d5f74ce5502a" (UID: "45b4afbd-39e3-4e51-a2ed-d5f74ce5502a"). InnerVolumeSpecName "kube-api-access-jxb9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:34:21.425474 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.425392    2573 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-config-file\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:34:21.425474 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.425421    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxb9w\" (UniqueName: \"kubernetes.io/projected/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a-kube-api-access-jxb9w\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:34:21.537985 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.537949    2573 generic.go:356] "Generic (PLEG): container finished" podID="45b4afbd-39e3-4e51-a2ed-d5f74ce5502a" containerID="29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0" exitCode=0
Sep 05 06:34:21.538148 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.537997    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"
Sep 05 06:34:21.538148 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.538033    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r" event={"ID":"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a","Type":"ContainerDied","Data":"29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0"}
Sep 05 06:34:21.538148 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.538093    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-mxj8r" event={"ID":"45b4afbd-39e3-4e51-a2ed-d5f74ce5502a","Type":"ContainerDied","Data":"85409dfafc8ad29e8cc33075cfb8c7353ed60b2dc1f1640b91f7be11490ab167"}
Sep 05 06:34:21.538148 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.538122    2573 scope.go:117] "RemoveContainer" containerID="29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0"
Sep 05 06:34:21.546906 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.546879    2573 scope.go:117] "RemoveContainer" containerID="29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0"
Sep 05 06:34:21.547163 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:34:21.547143    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0\": container with ID starting with 29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0 not found: ID does not exist" containerID="29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0"
Sep 05 06:34:21.547232 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.547172    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0"} err="failed to get container status \"29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0\": rpc error: code = NotFound desc = could not find container \"29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0\": container with ID starting with 29b1b3810a57639f6b7f225759735d974fd116499bd28f147ade10cfa39389f0 not found: ID does not exist"
Sep 05 06:34:21.561943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.561917    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"]
Sep 05 06:34:21.566237 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.566207    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-mxj8r"]
Sep 05 06:34:21.791727 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.791692    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:21.894005 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:21.893970    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:22.160294 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.160219    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45b4afbd-39e3-4e51-a2ed-d5f74ce5502a" path="/var/lib/kubelet/pods/45b4afbd-39e3-4e51-a2ed-d5f74ce5502a/volumes"
Sep 05 06:34:22.295270 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.295234    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"]
Sep 05 06:34:22.295719 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.295705    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45b4afbd-39e3-4e51-a2ed-d5f74ce5502a" containerName="limitador"
Sep 05 06:34:22.295763 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.295722    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="45b4afbd-39e3-4e51-a2ed-d5f74ce5502a" containerName="limitador"
Sep 05 06:34:22.295844 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.295833    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="45b4afbd-39e3-4e51-a2ed-d5f74ce5502a" containerName="limitador"
Sep 05 06:34:22.300561 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.300543    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"
Sep 05 06:34:22.304964 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.304940    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 05 06:34:22.305172 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.305152    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 06:34:22.305463 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.305445    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-hgtqt\""
Sep 05 06:34:22.305968 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.305953    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 05 06:34:22.319163 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.319137    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"]
Sep 05 06:34:22.332564 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.332530    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96lj7\" (UniqueName: \"kubernetes.io/projected/83279cb2-f0e0-4d8f-a811-cdcfede4dba6-kube-api-access-96lj7\") pod \"postgres-69fb4594cf-zzcxs\" (UID: \"83279cb2-f0e0-4d8f-a811-cdcfede4dba6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"
Sep 05 06:34:22.332999 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.332630    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/83279cb2-f0e0-4d8f-a811-cdcfede4dba6-data\") pod \"postgres-69fb4594cf-zzcxs\" (UID: \"83279cb2-f0e0-4d8f-a811-cdcfede4dba6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"
Sep 05 06:34:22.341766 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.341743    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:22.434113 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.434021    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96lj7\" (UniqueName: \"kubernetes.io/projected/83279cb2-f0e0-4d8f-a811-cdcfede4dba6-kube-api-access-96lj7\") pod \"postgres-69fb4594cf-zzcxs\" (UID: \"83279cb2-f0e0-4d8f-a811-cdcfede4dba6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"
Sep 05 06:34:22.434113 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.434090    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/83279cb2-f0e0-4d8f-a811-cdcfede4dba6-data\") pod \"postgres-69fb4594cf-zzcxs\" (UID: \"83279cb2-f0e0-4d8f-a811-cdcfede4dba6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"
Sep 05 06:34:22.434481 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.434461    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/83279cb2-f0e0-4d8f-a811-cdcfede4dba6-data\") pod \"postgres-69fb4594cf-zzcxs\" (UID: \"83279cb2-f0e0-4d8f-a811-cdcfede4dba6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"
Sep 05 06:34:22.443922 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.443894    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96lj7\" (UniqueName: \"kubernetes.io/projected/83279cb2-f0e0-4d8f-a811-cdcfede4dba6-kube-api-access-96lj7\") pod \"postgres-69fb4594cf-zzcxs\" (UID: \"83279cb2-f0e0-4d8f-a811-cdcfede4dba6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"
Sep 05 06:34:22.610509 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.610476    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"
Sep 05 06:34:22.880252 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:22.880229    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"]
Sep 05 06:34:22.882731 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:34:22.882705    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83279cb2_f0e0_4d8f_a811_cdcfede4dba6.slice/crio-1eb764b0a8020561ae77de16ab736e95a245b8a66be976a8286d1b2b7f8c6ee3 WatchSource:0}: Error finding container 1eb764b0a8020561ae77de16ab736e95a245b8a66be976a8286d1b2b7f8c6ee3: Status 404 returned error can't find the container with id 1eb764b0a8020561ae77de16ab736e95a245b8a66be976a8286d1b2b7f8c6ee3
Sep 05 06:34:23.552412 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:23.552323    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs" event={"ID":"83279cb2-f0e0-4d8f-a811-cdcfede4dba6","Type":"ContainerStarted","Data":"1eb764b0a8020561ae77de16ab736e95a245b8a66be976a8286d1b2b7f8c6ee3"}
Sep 05 06:34:28.588699 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:28.588654    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs" event={"ID":"83279cb2-f0e0-4d8f-a811-cdcfede4dba6","Type":"ContainerStarted","Data":"df4c196f31307b4eebd5aa24b5f4719bb7cf878586e01f3e4c2faed5706845d7"}
Sep 05 06:34:28.589132 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:28.588762    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"
Sep 05 06:34:28.609146 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:28.609103    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs" podStartSLOduration=1.938410524 podStartE2EDuration="6.609087381s" totalImagesPullingTime="4.670676857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:34:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:34:28.608039245 +0000 UTC m=+516.976086469" watchObservedRunningTime="2026-09-05 06:34:28.609087381 +0000 UTC m=+516.977134590"
Sep 05 06:34:34.623478 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:34.623448    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zzcxs"
Sep 05 06:34:34.990963 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:34.990930    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:35.094638 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:35.094588    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:44.208673 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.208582    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:44.211643 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.211620    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b669d7d5f-v7xsh"]
Sep 05 06:34:44.238997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.238961    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b669d7d5f-v7xsh"]
Sep 05 06:34:44.239174 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.239113    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:44.241629 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.241605    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4zf7f\""
Sep 05 06:34:44.241629 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.241627    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 05 06:34:44.241837 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.241769    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 05 06:34:44.242540 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.242523    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 05 06:34:44.283111 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.283066    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:44.335376 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.335328    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-webhook-cert\") pod \"maas-controller-6b669d7d5f-v7xsh\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") " pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:44.335547 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.335381    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmch6\" (UniqueName: \"kubernetes.io/projected/972f4262-4488-49cd-836a-22ef8157c923-kube-api-access-gmch6\") pod \"maas-controller-6b669d7d5f-v7xsh\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") " pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:44.335590 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.335552    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-metrics-certs\") pod \"maas-controller-6b669d7d5f-v7xsh\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") " pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:44.341828 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.341791    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:44.394835 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.394799    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-66bwj"]
Sep 05 06:34:44.398840 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.398822    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:44.406832 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.406808    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-66bwj"]
Sep 05 06:34:44.411979 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.411947    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:44.436153 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.436120    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-metrics-certs\") pod \"maas-controller-6b669d7d5f-v7xsh\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") " pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:44.436294 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.436178    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-webhook-cert\") pod \"maas-controller-6b669d7d5f-v7xsh\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") " pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:44.436294 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.436200    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmch6\" (UniqueName: \"kubernetes.io/projected/972f4262-4488-49cd-836a-22ef8157c923-kube-api-access-gmch6\") pod \"maas-controller-6b669d7d5f-v7xsh\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") " pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:44.438698 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.438671    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-webhook-cert\") pod \"maas-controller-6b669d7d5f-v7xsh\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") " pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:44.438811 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.438701    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-metrics-certs\") pod \"maas-controller-6b669d7d5f-v7xsh\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") " pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:44.445700 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.445671    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmch6\" (UniqueName: \"kubernetes.io/projected/972f4262-4488-49cd-836a-22ef8157c923-kube-api-access-gmch6\") pod \"maas-controller-6b669d7d5f-v7xsh\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") " pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:44.483988 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.483903    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:44.536968 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.536936    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-webhook-cert\") pod \"maas-controller-6bcd5999b5-66bwj\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") " pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:44.537154 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.536986    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-metrics-certs\") pod \"maas-controller-6bcd5999b5-66bwj\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") " pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:44.537154 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.537115    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5dvv\" (UniqueName: \"kubernetes.io/projected/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-kube-api-access-t5dvv\") pod \"maas-controller-6bcd5999b5-66bwj\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") " pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:44.550836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.550806    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:44.638252 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.638202    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5dvv\" (UniqueName: \"kubernetes.io/projected/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-kube-api-access-t5dvv\") pod \"maas-controller-6bcd5999b5-66bwj\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") " pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:44.638445 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.638281    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-webhook-cert\") pod \"maas-controller-6bcd5999b5-66bwj\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") " pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:44.638445 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.638326    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-metrics-certs\") pod \"maas-controller-6bcd5999b5-66bwj\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") " pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:44.640727 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.640704    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-metrics-certs\") pod \"maas-controller-6bcd5999b5-66bwj\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") " pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:44.640879 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.640732    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-webhook-cert\") pod \"maas-controller-6bcd5999b5-66bwj\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") " pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:44.647919 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.647891    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5dvv\" (UniqueName: \"kubernetes.io/projected/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-kube-api-access-t5dvv\") pod \"maas-controller-6bcd5999b5-66bwj\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") " pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:44.710759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.710718    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:44.824949 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.824924    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b669d7d5f-v7xsh"]
Sep 05 06:34:44.826907 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:34:44.826880    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod972f4262_4488_49cd_836a_22ef8157c923.slice/crio-ae3a561615e9c866ca3793e1fd6a5758dc285d2316ec9b8ff93f2aefaf4a32a7 WatchSource:0}: Error finding container ae3a561615e9c866ca3793e1fd6a5758dc285d2316ec9b8ff93f2aefaf4a32a7: Status 404 returned error can't find the container with id ae3a561615e9c866ca3793e1fd6a5758dc285d2316ec9b8ff93f2aefaf4a32a7
Sep 05 06:34:44.987470 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:44.987412    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-66bwj"]
Sep 05 06:34:44.990046 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:34:44.990020    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad2ceaaa_b8a4_4e08_adf6_2f9eff78ab6a.slice/crio-84ec2f742a7dea8ebefb84b9119a8febd4fc80ddf7f36fc1b7d96615ded4fb9d WatchSource:0}: Error finding container 84ec2f742a7dea8ebefb84b9119a8febd4fc80ddf7f36fc1b7d96615ded4fb9d: Status 404 returned error can't find the container with id 84ec2f742a7dea8ebefb84b9119a8febd4fc80ddf7f36fc1b7d96615ded4fb9d
Sep 05 06:34:45.672323 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:45.672283    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcd5999b5-66bwj" event={"ID":"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a","Type":"ContainerStarted","Data":"84ec2f742a7dea8ebefb84b9119a8febd4fc80ddf7f36fc1b7d96615ded4fb9d"}
Sep 05 06:34:45.674389 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:45.674295    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh" event={"ID":"972f4262-4488-49cd-836a-22ef8157c923","Type":"ContainerStarted","Data":"ae3a561615e9c866ca3793e1fd6a5758dc285d2316ec9b8ff93f2aefaf4a32a7"}
Sep 05 06:34:47.617116 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:47.617081    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:47.685758 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:47.685661    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcd5999b5-66bwj" event={"ID":"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a","Type":"ContainerStarted","Data":"d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de"}
Sep 05 06:34:47.685945 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:47.685836    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:47.687233 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:47.687211    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh" event={"ID":"972f4262-4488-49cd-836a-22ef8157c923","Type":"ContainerStarted","Data":"858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9"}
Sep 05 06:34:47.687337 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:47.687261    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:47.706939 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:47.706898    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bcd5999b5-66bwj" podStartSLOduration=1.313861648 podStartE2EDuration="3.706885493s" totalImagesPullingTime="2.393023845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:34:47.704994082 +0000 UTC m=+536.073041292" watchObservedRunningTime="2026-09-05 06:34:47.706885493 +0000 UTC m=+536.074932702"
Sep 05 06:34:47.710392 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:47.710366    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:47.727636 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:47.727592    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh" podStartSLOduration=1.18182076 podStartE2EDuration="3.727578731s" totalImagesPullingTime="2.545757971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:34:47.725599895 +0000 UTC m=+536.093647105" watchObservedRunningTime="2026-09-05 06:34:47.727578731 +0000 UTC m=+536.095625940"
Sep 05 06:34:55.350200 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:55.350164    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:55.411903 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:55.411832    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:34:58.693108 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:58.693079    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:58.693524 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:58.693269    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:34:58.771682 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:58.771641    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b669d7d5f-v7xsh"]
Sep 05 06:34:58.771938 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:58.771883    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh" podUID="972f4262-4488-49cd-836a-22ef8157c923" containerName="manager" containerID="cri-o://858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9" gracePeriod=30
Sep 05 06:34:59.015642 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.015615    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:59.064338 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.064305    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmch6\" (UniqueName: \"kubernetes.io/projected/972f4262-4488-49cd-836a-22ef8157c923-kube-api-access-gmch6\") pod \"972f4262-4488-49cd-836a-22ef8157c923\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") "
Sep 05 06:34:59.064502 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.064383    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-metrics-certs\") pod \"972f4262-4488-49cd-836a-22ef8157c923\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") "
Sep 05 06:34:59.064502 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.064415    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-webhook-cert\") pod \"972f4262-4488-49cd-836a-22ef8157c923\" (UID: \"972f4262-4488-49cd-836a-22ef8157c923\") "
Sep 05 06:34:59.066604 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.066569    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "972f4262-4488-49cd-836a-22ef8157c923" (UID: "972f4262-4488-49cd-836a-22ef8157c923"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:34:59.066604 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.066590    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/972f4262-4488-49cd-836a-22ef8157c923-kube-api-access-gmch6" (OuterVolumeSpecName: "kube-api-access-gmch6") pod "972f4262-4488-49cd-836a-22ef8157c923" (UID: "972f4262-4488-49cd-836a-22ef8157c923"). InnerVolumeSpecName "kube-api-access-gmch6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:34:59.066770 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.066584    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "972f4262-4488-49cd-836a-22ef8157c923" (UID: "972f4262-4488-49cd-836a-22ef8157c923"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:34:59.165037 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.164996    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:34:59.165037 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.165032    2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/972f4262-4488-49cd-836a-22ef8157c923-webhook-cert\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:34:59.165037 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.165045    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmch6\" (UniqueName: \"kubernetes.io/projected/972f4262-4488-49cd-836a-22ef8157c923-kube-api-access-gmch6\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:34:59.743177 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.743144    2573 generic.go:356] "Generic (PLEG): container finished" podID="972f4262-4488-49cd-836a-22ef8157c923" containerID="858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9" exitCode=0
Sep 05 06:34:59.743641 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.743193    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh"
Sep 05 06:34:59.743641 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.743194    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh" event={"ID":"972f4262-4488-49cd-836a-22ef8157c923","Type":"ContainerDied","Data":"858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9"}
Sep 05 06:34:59.743641 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.743334    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b669d7d5f-v7xsh" event={"ID":"972f4262-4488-49cd-836a-22ef8157c923","Type":"ContainerDied","Data":"ae3a561615e9c866ca3793e1fd6a5758dc285d2316ec9b8ff93f2aefaf4a32a7"}
Sep 05 06:34:59.743641 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.743374    2573 scope.go:117] "RemoveContainer" containerID="858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9"
Sep 05 06:34:59.752139 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.752123    2573 scope.go:117] "RemoveContainer" containerID="858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9"
Sep 05 06:34:59.752395 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:34:59.752378    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9\": container with ID starting with 858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9 not found: ID does not exist" containerID="858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9"
Sep 05 06:34:59.752451 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.752403    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9"} err="failed to get container status \"858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9\": rpc error: code = NotFound desc = could not find container \"858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9\": container with ID starting with 858d6349f8ce804d2e683b19de2350c2a5d4a5147ffe73a6c2f0a236eeb387d9 not found: ID does not exist"
Sep 05 06:34:59.767165 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.767136    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b669d7d5f-v7xsh"]
Sep 05 06:34:59.771742 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:34:59.771720    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b669d7d5f-v7xsh"]
Sep 05 06:35:00.155745 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:00.155666    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="972f4262-4488-49cd-836a-22ef8157c923" path="/var/lib/kubelet/pods/972f4262-4488-49cd-836a-22ef8157c923/volumes"
Sep 05 06:35:20.238156 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:20.238121    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:35:20.341808 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:20.341779    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:35:21.749541 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.749511    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:35:21.850093 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.850055    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:35:21.900367 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.900332    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"]
Sep 05 06:35:21.900835 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.900820    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="972f4262-4488-49cd-836a-22ef8157c923" containerName="manager"
Sep 05 06:35:21.900938 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.900838    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="972f4262-4488-49cd-836a-22ef8157c923" containerName="manager"
Sep 05 06:35:21.900994 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.900961    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="972f4262-4488-49cd-836a-22ef8157c923" containerName="manager"
Sep 05 06:35:21.907769 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.907751    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:21.910446 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.910420    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 05 06:35:21.910540 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.910450    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qdhh9\""
Sep 05 06:35:21.910540 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.910454    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 05 06:35:21.913926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.913891    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"]
Sep 05 06:35:21.969752 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.969719    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-metrics-certs\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:21.969943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.969778    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-maas-api-tls\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:21.969943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.969820    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8339082b-fd9e-41c6-b632-900dd2093133-openshift-service-ca\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:21.969943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:21.969884    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n8cx\" (UniqueName: \"kubernetes.io/projected/8339082b-fd9e-41c6-b632-900dd2093133-kube-api-access-8n8cx\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:22.071081 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.070995    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-maas-api-tls\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:22.071081 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.071043    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8339082b-fd9e-41c6-b632-900dd2093133-openshift-service-ca\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:22.071081 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.071062    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n8cx\" (UniqueName: \"kubernetes.io/projected/8339082b-fd9e-41c6-b632-900dd2093133-kube-api-access-8n8cx\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:22.071370 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.071107    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-metrics-certs\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:22.071370 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:35:22.071164    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 05 06:35:22.071370 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:35:22.071248    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-maas-api-tls podName:8339082b-fd9e-41c6-b632-900dd2093133 nodeName:}" failed. No retries permitted until 2026-09-05 06:35:22.571228772 +0000 UTC m=+570.939275973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-maas-api-tls") pod "maas-api-8457d65c99-nj7r8" (UID: "8339082b-fd9e-41c6-b632-900dd2093133") : secret "maas-api-serving-cert" not found
Sep 05 06:35:22.071710 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.071680    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8339082b-fd9e-41c6-b632-900dd2093133-openshift-service-ca\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:22.073733 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.073713    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-metrics-certs\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:22.080724 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.080701    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n8cx\" (UniqueName: \"kubernetes.io/projected/8339082b-fd9e-41c6-b632-900dd2093133-kube-api-access-8n8cx\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:22.094372 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.094346    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:35:22.195113 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.195075    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:35:22.324970 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.324877    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:35:22.442534 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.442501    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-sslnv"]
Sep 05 06:35:22.445984 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.445962    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv"
Sep 05 06:35:22.448475 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.448458    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 05 06:35:22.448599 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.448578    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pntf4\""
Sep 05 06:35:22.456240 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.456220    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-sslnv"]
Sep 05 06:35:22.551810 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.551781    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:35:22.574956 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.574921    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv9mj\" (UniqueName: \"kubernetes.io/projected/9a9386d4-aa77-4949-af89-303484782499-kube-api-access-bv9mj\") pod \"payload-processing-5b54b7fbb7-sslnv\" (UID: \"9a9386d4-aa77-4949-af89-303484782499\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv"
Sep 05 06:35:22.575153 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.574969    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-maas-api-tls\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:22.575153 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.574993    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a9386d4-aa77-4949-af89-303484782499-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-sslnv\" (UID: \"9a9386d4-aa77-4949-af89-303484782499\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv"
Sep 05 06:35:22.577436 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.577414    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-maas-api-tls\") pod \"maas-api-8457d65c99-nj7r8\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") " pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:22.676250 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.676210    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv9mj\" (UniqueName: \"kubernetes.io/projected/9a9386d4-aa77-4949-af89-303484782499-kube-api-access-bv9mj\") pod \"payload-processing-5b54b7fbb7-sslnv\" (UID: \"9a9386d4-aa77-4949-af89-303484782499\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv"
Sep 05 06:35:22.676430 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.676264    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a9386d4-aa77-4949-af89-303484782499-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-sslnv\" (UID: \"9a9386d4-aa77-4949-af89-303484782499\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv"
Sep 05 06:35:22.677052 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.677032    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a9386d4-aa77-4949-af89-303484782499-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-sslnv\" (UID: \"9a9386d4-aa77-4949-af89-303484782499\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv"
Sep 05 06:35:22.689257 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.689223    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv9mj\" (UniqueName: \"kubernetes.io/projected/9a9386d4-aa77-4949-af89-303484782499-kube-api-access-bv9mj\") pod \"payload-processing-5b54b7fbb7-sslnv\" (UID: \"9a9386d4-aa77-4949-af89-303484782499\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv"
Sep 05 06:35:22.757212 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.757183    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv"
Sep 05 06:35:22.820252 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:22.820220    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:23.029533 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:23.029499    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-sslnv"]
Sep 05 06:35:23.095721 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:23.095699    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"]
Sep 05 06:35:23.097513 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:35:23.097489    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8339082b_fd9e_41c6_b632_900dd2093133.slice/crio-00481e499fe49991624a4955434223c83702ae33554713e1296fa9e84e39faec WatchSource:0}: Error finding container 00481e499fe49991624a4955434223c83702ae33554713e1296fa9e84e39faec: Status 404 returned error can't find the container with id 00481e499fe49991624a4955434223c83702ae33554713e1296fa9e84e39faec
Sep 05 06:35:23.859760 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:23.859707    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8" event={"ID":"8339082b-fd9e-41c6-b632-900dd2093133","Type":"ContainerStarted","Data":"00481e499fe49991624a4955434223c83702ae33554713e1296fa9e84e39faec"}
Sep 05 06:35:23.861511 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:23.861482    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv" event={"ID":"9a9386d4-aa77-4949-af89-303484782499","Type":"ContainerStarted","Data":"d9388bc61473bcde6225fb2d6cd199b1592610b28cfe430c0d8b5751d69288e4"}
Sep 05 06:35:25.874003 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:25.873949    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8" event={"ID":"8339082b-fd9e-41c6-b632-900dd2093133","Type":"ContainerStarted","Data":"f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461"}
Sep 05 06:35:25.874469 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:25.874060    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:25.875514 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:25.875487    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv" event={"ID":"9a9386d4-aa77-4949-af89-303484782499","Type":"ContainerStarted","Data":"8dd0029ec9bdcbda637011a0eb3c8bcd5a38a66b126356956a4491fc3144b2be"}
Sep 05 06:35:25.875673 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:25.875620    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv"
Sep 05 06:35:25.897918 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:25.897860    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8" podStartSLOduration=2.9360839949999997 podStartE2EDuration="4.897830117s" totalImagesPullingTime="1.961746122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:35:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:35:25.895450911 +0000 UTC m=+574.263498124" watchObservedRunningTime="2026-09-05 06:35:25.897830117 +0000 UTC m=+574.265877328"
Sep 05 06:35:25.919358 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:25.919314    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv" podStartSLOduration=1.883773825 podStartE2EDuration="3.919298253s" totalImagesPullingTime="2.035524428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:35:25.917236726 +0000 UTC m=+574.285283936" watchObservedRunningTime="2026-09-05 06:35:25.919298253 +0000 UTC m=+574.287345508"
Sep 05 06:35:31.885926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:31.885896    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 06:35:32.943116 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:32.943082    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:35:33.037722 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:33.037662    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:35:36.883185 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:36.883146    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-sslnv"
Sep 05 06:35:44.620246 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:44.620217    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:35:52.099586 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:52.099557    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:35:52.100110 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:35:52.099562    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:36:02.591736 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:02.591704    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:36:11.332688 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:11.332652    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:36:15.392291 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:15.392203    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:36:20.597358 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.597326    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"]
Sep 05 06:36:20.600841 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.600824    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.604566 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.604542    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 05 06:36:20.604566 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.604555    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 05 06:36:20.604769 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.604544    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 05 06:36:20.604769 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.604546    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jfksv\""
Sep 05 06:36:20.611738 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.611716    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"]
Sep 05 06:36:20.619542 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.619514    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:36:20.681275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.681234    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.681275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.681282    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.681489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.681314    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls5wg\" (UniqueName: \"kubernetes.io/projected/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-kube-api-access-ls5wg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.681489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.681394    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.681489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.681426    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.681489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.681449    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.782805 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.782759    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls5wg\" (UniqueName: \"kubernetes.io/projected/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-kube-api-access-ls5wg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.782996 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.782907    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.782996 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.782936    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.782996 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.782961    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.782996 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.782995    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.783198 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.783021    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.783446 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.783415    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.783542 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.783457    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.783542 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.783502    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.785246 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.785216    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.785516 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.785497    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.792121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.792094    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls5wg\" (UniqueName: \"kubernetes.io/projected/ed79bed1-2aea-40e5-8a55-4bbcaf253c44-kube-api-access-ls5wg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x8vj\" (UID: \"ed79bed1-2aea-40e5-8a55-4bbcaf253c44\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:20.911207 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:20.911119    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:21.188661 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:21.188631    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"]
Sep 05 06:36:21.191458 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:36:21.191431    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded79bed1_2aea_40e5_8a55_4bbcaf253c44.slice/crio-2f61fbd237fe049b5d43a7fdd4489f8778b05f11306a1c116d77f5c9f4ea8fd1 WatchSource:0}: Error finding container 2f61fbd237fe049b5d43a7fdd4489f8778b05f11306a1c116d77f5c9f4ea8fd1: Status 404 returned error can't find the container with id 2f61fbd237fe049b5d43a7fdd4489f8778b05f11306a1c116d77f5c9f4ea8fd1
Sep 05 06:36:21.192658 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:21.192640    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 06:36:22.143696 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:22.143655    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj" event={"ID":"ed79bed1-2aea-40e5-8a55-4bbcaf253c44","Type":"ContainerStarted","Data":"2f61fbd237fe049b5d43a7fdd4489f8778b05f11306a1c116d77f5c9f4ea8fd1"}
Sep 05 06:36:23.149944 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:23.149911    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj" event={"ID":"ed79bed1-2aea-40e5-8a55-4bbcaf253c44","Type":"ContainerStarted","Data":"4c4d2ad3121e56bdd12289998a0d78a9150e888c783ba746814b1d171496d5a6"}
Sep 05 06:36:23.170402 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:23.170354    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj" podStartSLOduration=2.192450879 podStartE2EDuration="3.170337214s" totalImagesPullingTime="977.886335ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:36:23.169637538 +0000 UTC m=+631.537684749" watchObservedRunningTime="2026-09-05 06:36:23.170337214 +0000 UTC m=+631.538384460"
Sep 05 06:36:27.731098 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:27.731052    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:36:27.828655 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:27.828620    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:36:30.912078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:30.912044    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:30.912449 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:30.912104    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:30.924419 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:30.924390    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:31.196064 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:31.195974    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x8vj"
Sep 05 06:36:36.469081 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:36.469050    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:36:50.348941 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:50.348877    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:36:56.305665 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:36:56.305627    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:37:06.712355 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:37:06.712320    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:10.149723 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:10.149625    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:11.398412 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:11.398374    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:17.772619 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:17.772585    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:27.980645 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:27.980596    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:28.092555 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:28.092530    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:28.494101 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:28.494065    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:28.592083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:28.592043    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:29.531910 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:29.531881    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:29.631907 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:29.631876    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:30.044943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:30.044909    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:30.115266 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:30.115233    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7750-139898255617072"]
Sep 05 06:38:30.118606 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:30.118588    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139898255617072"
Sep 05 06:38:30.121249 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:30.121216    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wwpm9\""
Sep 05 06:38:30.123633 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:30.123613    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139898255617072"]
Sep 05 06:38:30.254178 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:30.254136    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrbrv\" (UniqueName: \"kubernetes.io/projected/16418a74-a467-4126-9785-6ee30579b984-kube-api-access-xrbrv\") pod \"test-curl-7750-139898255617072\" (UID: \"16418a74-a467-4126-9785-6ee30579b984\") " pod="opendatahub/test-curl-7750-139898255617072"
Sep 05 06:38:30.355682 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:30.355602    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrbrv\" (UniqueName: \"kubernetes.io/projected/16418a74-a467-4126-9785-6ee30579b984-kube-api-access-xrbrv\") pod \"test-curl-7750-139898255617072\" (UID: \"16418a74-a467-4126-9785-6ee30579b984\") " pod="opendatahub/test-curl-7750-139898255617072"
Sep 05 06:38:30.365048 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:30.365025    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrbrv\" (UniqueName: \"kubernetes.io/projected/16418a74-a467-4126-9785-6ee30579b984-kube-api-access-xrbrv\") pod \"test-curl-7750-139898255617072\" (UID: \"16418a74-a467-4126-9785-6ee30579b984\") " pod="opendatahub/test-curl-7750-139898255617072"
Sep 05 06:38:30.429079 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:30.429043    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139898255617072"
Sep 05 06:38:30.710587 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:30.710436    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139898255617072"]
Sep 05 06:38:30.712740 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:38:30.712715    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16418a74_a467_4126_9785_6ee30579b984.slice/crio-2c7587aa7536335c4f295b15ad8ed36c862a0ef50371ecbd1bf21e36fac19b7f WatchSource:0}: Error finding container 2c7587aa7536335c4f295b15ad8ed36c862a0ef50371ecbd1bf21e36fac19b7f: Status 404 returned error can't find the container with id 2c7587aa7536335c4f295b15ad8ed36c862a0ef50371ecbd1bf21e36fac19b7f
Sep 05 06:38:30.736899 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:30.736840    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139898255617072" event={"ID":"16418a74-a467-4126-9785-6ee30579b984","Type":"ContainerStarted","Data":"2c7587aa7536335c4f295b15ad8ed36c862a0ef50371ecbd1bf21e36fac19b7f"}
Sep 05 06:38:31.672402 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:31.672370    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:31.743072 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:31.743041    2573 generic.go:356] "Generic (PLEG): container finished" podID="16418a74-a467-4126-9785-6ee30579b984" containerID="df56e34a9a11eb8dc8fe022a7daf8da5820a88469244976659de6b4afe3d79c1" exitCode=0
Sep 05 06:38:31.743466 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:31.743126    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139898255617072" event={"ID":"16418a74-a467-4126-9785-6ee30579b984","Type":"ContainerDied","Data":"df56e34a9a11eb8dc8fe022a7daf8da5820a88469244976659de6b4afe3d79c1"}
Sep 05 06:38:31.775071 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:31.775027    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:32.879716 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:32.879686    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139898255617072"
Sep 05 06:38:32.903862 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:32.903826    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7750-139898255617072_16418a74-a467-4126-9785-6ee30579b984/test-curl-7750-139898255617072/0.log"
Sep 05 06:38:32.929398 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:32.929365    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7750-139898255617072"]
Sep 05 06:38:32.935940 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:32.935915    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7750-139898255617072"]
Sep 05 06:38:32.980270 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:32.980239    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrbrv\" (UniqueName: \"kubernetes.io/projected/16418a74-a467-4126-9785-6ee30579b984-kube-api-access-xrbrv\") pod \"16418a74-a467-4126-9785-6ee30579b984\" (UID: \"16418a74-a467-4126-9785-6ee30579b984\") "
Sep 05 06:38:32.982751 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:32.982721    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16418a74-a467-4126-9785-6ee30579b984-kube-api-access-xrbrv" (OuterVolumeSpecName: "kube-api-access-xrbrv") pod "16418a74-a467-4126-9785-6ee30579b984" (UID: "16418a74-a467-4126-9785-6ee30579b984"). InnerVolumeSpecName "kube-api-access-xrbrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:38:33.000343 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.000263    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"]
Sep 05 06:38:33.001031 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.000894    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16418a74-a467-4126-9785-6ee30579b984" containerName="test-curl-7750-139898255617072"
Sep 05 06:38:33.001031 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.000914    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="16418a74-a467-4126-9785-6ee30579b984" containerName="test-curl-7750-139898255617072"
Sep 05 06:38:33.001031 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.001001    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="16418a74-a467-4126-9785-6ee30579b984" containerName="test-curl-7750-139898255617072"
Sep 05 06:38:33.004447 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.004424    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.006939 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.006904    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-45e697-serving-cert\""
Sep 05 06:38:33.015333 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.015303    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"]
Sep 05 06:38:33.034920 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.034895    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:33.057715 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.057687    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7750-139898255617328"]
Sep 05 06:38:33.061110 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.061091    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139898255617328"
Sep 05 06:38:33.074793 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.074767    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139898255617328"]
Sep 05 06:38:33.082319 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.082285    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrbrv\" (UniqueName: \"kubernetes.io/projected/16418a74-a467-4126-9785-6ee30579b984-kube-api-access-xrbrv\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:38:33.136439 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.136403    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:33.183192 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.183151    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29fsw\" (UniqueName: \"kubernetes.io/projected/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-kube-api-access-29fsw\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.183369 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.183221    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-maas-api-tls\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.183369 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.183243    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-metrics-certs\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.183369 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.183294    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhgpl\" (UniqueName: \"kubernetes.io/projected/59a6fee1-0ca7-4dfd-a32a-00a71f79401e-kube-api-access-hhgpl\") pod \"test-curl-7750-139898255617328\" (UID: \"59a6fee1-0ca7-4dfd-a32a-00a71f79401e\") " pod="opendatahub/test-curl-7750-139898255617328"
Sep 05 06:38:33.183491 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.183386    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-openshift-service-ca\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.284387 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.284308    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgpl\" (UniqueName: \"kubernetes.io/projected/59a6fee1-0ca7-4dfd-a32a-00a71f79401e-kube-api-access-hhgpl\") pod \"test-curl-7750-139898255617328\" (UID: \"59a6fee1-0ca7-4dfd-a32a-00a71f79401e\") " pod="opendatahub/test-curl-7750-139898255617328"
Sep 05 06:38:33.284387 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.284354    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-openshift-service-ca\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.284387 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.284383    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29fsw\" (UniqueName: \"kubernetes.io/projected/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-kube-api-access-29fsw\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.284686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.284436    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-maas-api-tls\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.284686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.284454    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-metrics-certs\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.284686 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:38:33.284569    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-serving-cert: secret "maas-api-e2e-inf-a-45e697-serving-cert" not found
Sep 05 06:38:33.284686 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:38:33.284636    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-maas-api-tls podName:ba3a1c79-f1ba-4f64-9e91-9c2a66277d95 nodeName:}" failed. No retries permitted until 2026-09-05 06:38:33.784618967 +0000 UTC m=+762.152666159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-maas-api-tls") pod "maas-api-e2e-inf-a-45e697-79c967cd49-ltk85" (UID: "ba3a1c79-f1ba-4f64-9e91-9c2a66277d95") : secret "maas-api-e2e-inf-a-45e697-serving-cert" not found
Sep 05 06:38:33.285127 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.285103    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-openshift-service-ca\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.286990 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.286967    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-metrics-certs\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.293628 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.293606    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhgpl\" (UniqueName: \"kubernetes.io/projected/59a6fee1-0ca7-4dfd-a32a-00a71f79401e-kube-api-access-hhgpl\") pod \"test-curl-7750-139898255617328\" (UID: \"59a6fee1-0ca7-4dfd-a32a-00a71f79401e\") " pod="opendatahub/test-curl-7750-139898255617328"
Sep 05 06:38:33.294195 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.294175    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29fsw\" (UniqueName: \"kubernetes.io/projected/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-kube-api-access-29fsw\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.372415 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.372385    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139898255617328"
Sep 05 06:38:33.511438 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.511397    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:33.610207 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.610121    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:33.646114 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.645933    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139898255617328"]
Sep 05 06:38:33.648309 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:38:33.648283    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59a6fee1_0ca7_4dfd_a32a_00a71f79401e.slice/crio-f125190b493b9d0321a9ba1fc9ce12777823f070b4dfeaeed7e57c367a6add08 WatchSource:0}: Error finding container f125190b493b9d0321a9ba1fc9ce12777823f070b4dfeaeed7e57c367a6add08: Status 404 returned error can't find the container with id f125190b493b9d0321a9ba1fc9ce12777823f070b4dfeaeed7e57c367a6add08
Sep 05 06:38:33.688327 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.688294    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:33.754867 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.754833    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c7587aa7536335c4f295b15ad8ed36c862a0ef50371ecbd1bf21e36fac19b7f"
Sep 05 06:38:33.754867 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.754835    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139898255617072"
Sep 05 06:38:33.756054 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.756025    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139898255617328" event={"ID":"59a6fee1-0ca7-4dfd-a32a-00a71f79401e","Type":"ContainerStarted","Data":"f125190b493b9d0321a9ba1fc9ce12777823f070b4dfeaeed7e57c367a6add08"}
Sep 05 06:38:33.765651 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:38:33.765626    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7750-139898255617072\" is forbidden: User \"system:node:ip-10-0-128-174.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-174.ec2.internal' and this object" podUID="16418a74-a467-4126-9785-6ee30579b984" pod="opendatahub/test-curl-7750-139898255617072"
Sep 05 06:38:33.789134 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.789103    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-maas-api-tls\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.791491 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.791464    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-maas-api-tls\") pod \"maas-api-e2e-inf-a-45e697-79c967cd49-ltk85\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:33.917021 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:33.916934    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:34.155742 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:34.155708    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16418a74-a467-4126-9785-6ee30579b984" path="/var/lib/kubelet/pods/16418a74-a467-4126-9785-6ee30579b984/volumes"
Sep 05 06:38:34.192387 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:34.192359    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"]
Sep 05 06:38:34.194846 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:38:34.194818    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba3a1c79_f1ba_4f64_9e91_9c2a66277d95.slice/crio-c67af4436896057f492452c589bd7ebe4f553293780dbbb29e28b075b670e17f WatchSource:0}: Error finding container c67af4436896057f492452c589bd7ebe4f553293780dbbb29e28b075b670e17f: Status 404 returned error can't find the container with id c67af4436896057f492452c589bd7ebe4f553293780dbbb29e28b075b670e17f
Sep 05 06:38:34.762885 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:34.762838    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85" event={"ID":"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95","Type":"ContainerStarted","Data":"e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e"}
Sep 05 06:38:34.763078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:34.762913    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85" event={"ID":"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95","Type":"ContainerStarted","Data":"c67af4436896057f492452c589bd7ebe4f553293780dbbb29e28b075b670e17f"}
Sep 05 06:38:34.763078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:34.762945    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:34.764413 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:34.764383    2573 generic.go:356] "Generic (PLEG): container finished" podID="59a6fee1-0ca7-4dfd-a32a-00a71f79401e" containerID="a5553f9a229a3d9e151270b5a664ce437e41012b98e462c929cb54424781e1fb" exitCode=0
Sep 05 06:38:34.764551 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:34.764420    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139898255617328" event={"ID":"59a6fee1-0ca7-4dfd-a32a-00a71f79401e","Type":"ContainerDied","Data":"a5553f9a229a3d9e151270b5a664ce437e41012b98e462c929cb54424781e1fb"}
Sep 05 06:38:34.783286 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:34.783242    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85" podStartSLOduration=2.351370949 podStartE2EDuration="2.783226973s" totalImagesPullingTime="431.856024ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:38:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:38:34.782136796 +0000 UTC m=+763.150184007" watchObservedRunningTime="2026-09-05 06:38:34.783226973 +0000 UTC m=+763.151274182"
Sep 05 06:38:35.896076 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:35.896049    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139898255617328"
Sep 05 06:38:35.904707 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:35.904687    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhgpl\" (UniqueName: \"kubernetes.io/projected/59a6fee1-0ca7-4dfd-a32a-00a71f79401e-kube-api-access-hhgpl\") pod \"59a6fee1-0ca7-4dfd-a32a-00a71f79401e\" (UID: \"59a6fee1-0ca7-4dfd-a32a-00a71f79401e\") "
Sep 05 06:38:35.906589 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:35.906562    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59a6fee1-0ca7-4dfd-a32a-00a71f79401e-kube-api-access-hhgpl" (OuterVolumeSpecName: "kube-api-access-hhgpl") pod "59a6fee1-0ca7-4dfd-a32a-00a71f79401e" (UID: "59a6fee1-0ca7-4dfd-a32a-00a71f79401e"). InnerVolumeSpecName "kube-api-access-hhgpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:38:35.916846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:35.916827    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7750-139898255617328_59a6fee1-0ca7-4dfd-a32a-00a71f79401e/test-curl-7750-139898255617328/0.log"
Sep 05 06:38:35.941019 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:35.940990    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7750-139898255617328"]
Sep 05 06:38:35.947512 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:35.947492    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7750-139898255617328"]
Sep 05 06:38:36.005253 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:36.005216    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhgpl\" (UniqueName: \"kubernetes.io/projected/59a6fee1-0ca7-4dfd-a32a-00a71f79401e-kube-api-access-hhgpl\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:38:36.157462 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:36.157388    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59a6fee1-0ca7-4dfd-a32a-00a71f79401e" path="/var/lib/kubelet/pods/59a6fee1-0ca7-4dfd-a32a-00a71f79401e/volumes"
Sep 05 06:38:36.342817 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:36.342776    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:36.441807 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:36.441715    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:36.542222 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:36.542179    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:36.641947 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:36.641909    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:36.775390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:36.775363    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139898255617328"
Sep 05 06:38:36.775544 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:36.775413    2573 scope.go:117] "RemoveContainer" containerID="a5553f9a229a3d9e151270b5a664ce437e41012b98e462c929cb54424781e1fb"
Sep 05 06:38:36.896632 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:36.896577    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:36.992458 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:36.992419    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:37.053406 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:37.053310    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:37.773949 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:37.773914    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:40.112467 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:40.112433    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:40.211230 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:40.211190    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:40.310240 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:40.310206    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:40.774818 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:40.774791    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:38:41.250740 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.250705    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"]
Sep 05 06:38:41.251288 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.251274    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59a6fee1-0ca7-4dfd-a32a-00a71f79401e" containerName="test-curl-7750-139898255617328"
Sep 05 06:38:41.251347 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.251293    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="59a6fee1-0ca7-4dfd-a32a-00a71f79401e" containerName="test-curl-7750-139898255617328"
Sep 05 06:38:41.251402 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.251380    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="59a6fee1-0ca7-4dfd-a32a-00a71f79401e" containerName="test-curl-7750-139898255617328"
Sep 05 06:38:41.256125 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.256108    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.259086 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.259067    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b1d1c87d-serving-cert\""
Sep 05 06:38:41.268550 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.268525    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"]
Sep 05 06:38:41.291458 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.291425    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:41.350448 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.350413    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/074a3307-fbb5-48e2-9166-ec03ce6c3d88-openshift-service-ca\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.350607 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.350510    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwhvr\" (UniqueName: \"kubernetes.io/projected/074a3307-fbb5-48e2-9166-ec03ce6c3d88-kube-api-access-wwhvr\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.350607 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.350537    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-metrics-certs\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.350607 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.350581    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-maas-api-tls\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.390528 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.390500    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:41.451211 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.451179    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-maas-api-tls\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.451390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.451224    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/074a3307-fbb5-48e2-9166-ec03ce6c3d88-openshift-service-ca\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.451390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.451279    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwhvr\" (UniqueName: \"kubernetes.io/projected/074a3307-fbb5-48e2-9166-ec03ce6c3d88-kube-api-access-wwhvr\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.451390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.451309    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-metrics-certs\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.451390 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:38:41.451341    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-serving-cert: secret "maas-api-e2e-ait-b1d1c87d-serving-cert" not found
Sep 05 06:38:41.451645 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:38:41.451435    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-maas-api-tls podName:074a3307-fbb5-48e2-9166-ec03ce6c3d88 nodeName:}" failed. No retries permitted until 2026-09-05 06:38:41.951414332 +0000 UTC m=+770.319461536 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-maas-api-tls") pod "maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm" (UID: "074a3307-fbb5-48e2-9166-ec03ce6c3d88") : secret "maas-api-e2e-ait-b1d1c87d-serving-cert" not found
Sep 05 06:38:41.452015 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.451993    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/074a3307-fbb5-48e2-9166-ec03ce6c3d88-openshift-service-ca\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.453760 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.453735    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-metrics-certs\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.460356 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.460332    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwhvr\" (UniqueName: \"kubernetes.io/projected/074a3307-fbb5-48e2-9166-ec03ce6c3d88-kube-api-access-wwhvr\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.587496 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.587410    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:41.849745 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.849656    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:41.943906 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.943871    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:41.956805 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.956767    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-maas-api-tls\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:41.959240 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:41.959217    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-maas-api-tls\") pod \"maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:42.030923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.030893    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"]
Sep 05 06:38:42.034708 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.034690    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:38:42.038906 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.038886    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b1d1c87d-dockercfg-sdqzx\""
Sep 05 06:38:42.038978 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.038912    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b1d1c87d\""
Sep 05 06:38:42.050460 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.050435    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"]
Sep 05 06:38:42.057474 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.057448    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcw6d\" (UniqueName: \"kubernetes.io/projected/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-kube-api-access-tcw6d\") pod \"payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf\" (UID: \"bf443cfb-2bfc-468d-a9b1-efff2780b1d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:38:42.057580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.057509    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf\" (UID: \"bf443cfb-2bfc-468d-a9b1-efff2780b1d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:38:42.150157 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.150087    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"]
Sep 05 06:38:42.153672 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.153649    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:38:42.158022 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.158001    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf\" (UID: \"bf443cfb-2bfc-468d-a9b1-efff2780b1d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:38:42.158142 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.158101    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcw6d\" (UniqueName: \"kubernetes.io/projected/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-kube-api-access-tcw6d\") pod \"payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf\" (UID: \"bf443cfb-2bfc-468d-a9b1-efff2780b1d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:38:42.158713 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.158695    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf\" (UID: \"bf443cfb-2bfc-468d-a9b1-efff2780b1d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:38:42.165522 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.165499    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"]
Sep 05 06:38:42.167785 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.167767    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:42.171305 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.171285    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcw6d\" (UniqueName: \"kubernetes.io/projected/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-kube-api-access-tcw6d\") pod \"payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf\" (UID: \"bf443cfb-2bfc-468d-a9b1-efff2780b1d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:38:42.259247 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.259163    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pznjg\" (UniqueName: \"kubernetes.io/projected/649146e9-4424-4c5d-90df-cde530f16a71-kube-api-access-pznjg\") pod \"payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5\" (UID: \"649146e9-4424-4c5d-90df-cde530f16a71\") " pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:38:42.259247 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.259223    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/649146e9-4424-4c5d-90df-cde530f16a71-plugins-config-volume\") pod \"payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5\" (UID: \"649146e9-4424-4c5d-90df-cde530f16a71\") " pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:38:42.321797 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.321770    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"]
Sep 05 06:38:42.327058 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.327031    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.330341 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.330319    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-8956b5-serving-cert\""
Sep 05 06:38:42.338072 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.337616    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"]
Sep 05 06:38:42.344907 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.344883    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:38:42.360408 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.360383    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pznjg\" (UniqueName: \"kubernetes.io/projected/649146e9-4424-4c5d-90df-cde530f16a71-kube-api-access-pznjg\") pod \"payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5\" (UID: \"649146e9-4424-4c5d-90df-cde530f16a71\") " pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:38:42.360494 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.360445    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/649146e9-4424-4c5d-90df-cde530f16a71-plugins-config-volume\") pod \"payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5\" (UID: \"649146e9-4424-4c5d-90df-cde530f16a71\") " pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:38:42.360494 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.360476    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e50f8853-6b18-4c60-94df-a1769d459c2b-openshift-service-ca\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.360573 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.360525    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-metrics-certs\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.360573 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.360553    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc9p7\" (UniqueName: \"kubernetes.io/projected/e50f8853-6b18-4c60-94df-a1769d459c2b-kube-api-access-rc9p7\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.360651 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.360593    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.361423 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.361398    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/649146e9-4424-4c5d-90df-cde530f16a71-plugins-config-volume\") pod \"payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5\" (UID: \"649146e9-4424-4c5d-90df-cde530f16a71\") " pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:38:42.362087 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.362064    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:42.369727 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.369703    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pznjg\" (UniqueName: \"kubernetes.io/projected/649146e9-4424-4c5d-90df-cde530f16a71-kube-api-access-pznjg\") pod \"payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5\" (UID: \"649146e9-4424-4c5d-90df-cde530f16a71\") " pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:38:42.447455 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.447430    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"]
Sep 05 06:38:42.448773 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:38:42.448743    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod074a3307_fbb5_48e2_9166_ec03ce6c3d88.slice/crio-00bc001227eebf93d8dd9bcec80b7a0eb5cfbc0803c56c5360bc15262dbfd902 WatchSource:0}: Error finding container 00bc001227eebf93d8dd9bcec80b7a0eb5cfbc0803c56c5360bc15262dbfd902: Status 404 returned error can't find the container with id 00bc001227eebf93d8dd9bcec80b7a0eb5cfbc0803c56c5360bc15262dbfd902
Sep 05 06:38:42.449816 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.449778    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:42.462052 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.462026    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-metrics-certs\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.462148 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.462071    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc9p7\" (UniqueName: \"kubernetes.io/projected/e50f8853-6b18-4c60-94df-a1769d459c2b-kube-api-access-rc9p7\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.462148 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.462114    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.462262 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.462212    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e50f8853-6b18-4c60-94df-a1769d459c2b-openshift-service-ca\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.462377 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:38:42.462352    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-serving-cert: secret "maas-api-e2e-autores-8956b5-serving-cert" not found
Sep 05 06:38:42.462503 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:38:42.462427    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls podName:e50f8853-6b18-4c60-94df-a1769d459c2b nodeName:}" failed. No retries permitted until 2026-09-05 06:38:42.962406382 +0000 UTC m=+771.330453572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls") pod "maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb" (UID: "e50f8853-6b18-4c60-94df-a1769d459c2b") : secret "maas-api-e2e-autores-8956b5-serving-cert" not found
Sep 05 06:38:42.462973 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.462953    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e50f8853-6b18-4c60-94df-a1769d459c2b-openshift-service-ca\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.464099 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.464080    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:38:42.464453 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.464432    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-metrics-certs\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.473062 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.473041    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc9p7\" (UniqueName: \"kubernetes.io/projected/e50f8853-6b18-4c60-94df-a1769d459c2b-kube-api-access-rc9p7\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.624313 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.624288    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"]
Sep 05 06:38:42.626421 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:38:42.626391    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf443cfb_2bfc_468d_a9b1_efff2780b1d4.slice/crio-44e55b57cc221ce8a3bb068821835abee93166df50870015633818cab043ece3 WatchSource:0}: Error finding container 44e55b57cc221ce8a3bb068821835abee93166df50870015633818cab043ece3: Status 404 returned error can't find the container with id 44e55b57cc221ce8a3bb068821835abee93166df50870015633818cab043ece3
Sep 05 06:38:42.744701 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.744496    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"]
Sep 05 06:38:42.747232 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:38:42.747204    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod649146e9_4424_4c5d_90df_cde530f16a71.slice/crio-a643cb1305b37a28b8a7b3f24645d8efee85bbff9d95b9a9fe746ebce33e19fe WatchSource:0}: Error finding container a643cb1305b37a28b8a7b3f24645d8efee85bbff9d95b9a9fe746ebce33e19fe: Status 404 returned error can't find the container with id a643cb1305b37a28b8a7b3f24645d8efee85bbff9d95b9a9fe746ebce33e19fe
Sep 05 06:38:42.808657 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.808625    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5" event={"ID":"649146e9-4424-4c5d-90df-cde530f16a71","Type":"ContainerStarted","Data":"a643cb1305b37a28b8a7b3f24645d8efee85bbff9d95b9a9fe746ebce33e19fe"}
Sep 05 06:38:42.810521 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.810468    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf" event={"ID":"bf443cfb-2bfc-468d-a9b1-efff2780b1d4","Type":"ContainerStarted","Data":"3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c"}
Sep 05 06:38:42.810633 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.810531    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf" event={"ID":"bf443cfb-2bfc-468d-a9b1-efff2780b1d4","Type":"ContainerStarted","Data":"44e55b57cc221ce8a3bb068821835abee93166df50870015633818cab043ece3"}
Sep 05 06:38:42.810697 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.810648    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:38:42.811817 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.811792    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm" event={"ID":"074a3307-fbb5-48e2-9166-ec03ce6c3d88","Type":"ContainerStarted","Data":"00bc001227eebf93d8dd9bcec80b7a0eb5cfbc0803c56c5360bc15262dbfd902"}
Sep 05 06:38:42.831318 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.831275    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf" podStartSLOduration=0.831260547 podStartE2EDuration="831.260547ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:38:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:38:42.828880782 +0000 UTC m=+771.196927991" watchObservedRunningTime="2026-09-05 06:38:42.831260547 +0000 UTC m=+771.199307753"
Sep 05 06:38:42.894770 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.894740    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:42.966514 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.966410    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:42.966681 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:38:42.966596    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-serving-cert: secret "maas-api-e2e-autores-8956b5-serving-cert" not found
Sep 05 06:38:42.966681 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:38:42.966678    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls podName:e50f8853-6b18-4c60-94df-a1769d459c2b nodeName:}" failed. No retries permitted until 2026-09-05 06:38:43.966656272 +0000 UTC m=+772.334703473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls") pod "maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb" (UID: "e50f8853-6b18-4c60-94df-a1769d459c2b") : secret "maas-api-e2e-autores-8956b5-serving-cert" not found
Sep 05 06:38:42.994669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:42.994609    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:43.038724 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.038696    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"]
Sep 05 06:38:43.042804 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.042782    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:38:43.045790 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.045764    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-8956b5-dockercfg-2nmpw\""
Sep 05 06:38:43.045956 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.045819    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-8956b5\""
Sep 05 06:38:43.057154 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.057128    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"]
Sep 05 06:38:43.067309 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.067286    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp7vl\" (UniqueName: \"kubernetes.io/projected/0593df1b-a65b-4445-ab71-4e71cc10bd00-kube-api-access-dp7vl\") pod \"payload-processing-e2e-autores-8956b5-846947d97f-w28rc\" (UID: \"0593df1b-a65b-4445-ab71-4e71cc10bd00\") " pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:38:43.067426 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.067382    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0593df1b-a65b-4445-ab71-4e71cc10bd00-plugins-config-volume\") pod \"payload-processing-e2e-autores-8956b5-846947d97f-w28rc\" (UID: \"0593df1b-a65b-4445-ab71-4e71cc10bd00\") " pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:38:43.115504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.115471    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:43.168021 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.167986    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0593df1b-a65b-4445-ab71-4e71cc10bd00-plugins-config-volume\") pod \"payload-processing-e2e-autores-8956b5-846947d97f-w28rc\" (UID: \"0593df1b-a65b-4445-ab71-4e71cc10bd00\") " pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:38:43.168193 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.168082    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp7vl\" (UniqueName: \"kubernetes.io/projected/0593df1b-a65b-4445-ab71-4e71cc10bd00-kube-api-access-dp7vl\") pod \"payload-processing-e2e-autores-8956b5-846947d97f-w28rc\" (UID: \"0593df1b-a65b-4445-ab71-4e71cc10bd00\") " pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:38:43.168834 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.168813    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0593df1b-a65b-4445-ab71-4e71cc10bd00-plugins-config-volume\") pod \"payload-processing-e2e-autores-8956b5-846947d97f-w28rc\" (UID: \"0593df1b-a65b-4445-ab71-4e71cc10bd00\") " pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:38:43.185869 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.183689    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp7vl\" (UniqueName: \"kubernetes.io/projected/0593df1b-a65b-4445-ab71-4e71cc10bd00-kube-api-access-dp7vl\") pod \"payload-processing-e2e-autores-8956b5-846947d97f-w28rc\" (UID: \"0593df1b-a65b-4445-ab71-4e71cc10bd00\") " pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:38:43.357162 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.357124    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:38:43.426390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.426356    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:43.630769 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.630745    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"]
Sep 05 06:38:43.633343 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:38:43.633309    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0593df1b_a65b_4445_ab71_4e71cc10bd00.slice/crio-c5449f50a5b10737ddfba303b76fc703b693ab446a655e315b8831253a056a12 WatchSource:0}: Error finding container c5449f50a5b10737ddfba303b76fc703b693ab446a655e315b8831253a056a12: Status 404 returned error can't find the container with id c5449f50a5b10737ddfba303b76fc703b693ab446a655e315b8831253a056a12
Sep 05 06:38:43.818435 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.818400    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm" event={"ID":"074a3307-fbb5-48e2-9166-ec03ce6c3d88","Type":"ContainerStarted","Data":"44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed"}
Sep 05 06:38:43.818620 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.818541    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:43.820194 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.820168    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5" event={"ID":"649146e9-4424-4c5d-90df-cde530f16a71","Type":"ContainerStarted","Data":"eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea"}
Sep 05 06:38:43.820591 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.820559    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:38:43.822289 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.822237    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc" event={"ID":"0593df1b-a65b-4445-ab71-4e71cc10bd00","Type":"ContainerStarted","Data":"6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced"}
Sep 05 06:38:43.822391 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.822301    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc" event={"ID":"0593df1b-a65b-4445-ab71-4e71cc10bd00","Type":"ContainerStarted","Data":"c5449f50a5b10737ddfba303b76fc703b693ab446a655e315b8831253a056a12"}
Sep 05 06:38:43.822491 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.822466    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:38:43.837297 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.837231    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm" podStartSLOduration=2.472385894 podStartE2EDuration="2.837216607s" totalImagesPullingTime="364.830713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:38:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:38:43.835807094 +0000 UTC m=+772.203854303" watchObservedRunningTime="2026-09-05 06:38:43.837216607 +0000 UTC m=+772.205263812"
Sep 05 06:38:43.852256 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.852211    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc" podStartSLOduration=0.852197108 podStartE2EDuration="852.197108ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:38:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:38:43.852108303 +0000 UTC m=+772.220155513" watchObservedRunningTime="2026-09-05 06:38:43.852197108 +0000 UTC m=+772.220244316"
Sep 05 06:38:43.867268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.867223    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5" podStartSLOduration=1.867206572 podStartE2EDuration="1.867206572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:38:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:38:43.867183979 +0000 UTC m=+772.235231189" watchObservedRunningTime="2026-09-05 06:38:43.867206572 +0000 UTC m=+772.235253787"
Sep 05 06:38:43.976913 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.976797    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:43.979304 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:43.979270    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls\") pod \"maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:44.143159 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:44.143116    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:44.172170 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:44.172138    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:44.423384 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:44.423354    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"]
Sep 05 06:38:44.425676 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:38:44.425649    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50f8853_6b18_4c60_94df_a1769d459c2b.slice/crio-4e09bde10cfaff8bbbc329f0fda0371e6225ddf45523d4f147a53064cfd7a6bc WatchSource:0}: Error finding container 4e09bde10cfaff8bbbc329f0fda0371e6225ddf45523d4f147a53064cfd7a6bc: Status 404 returned error can't find the container with id 4e09bde10cfaff8bbbc329f0fda0371e6225ddf45523d4f147a53064cfd7a6bc
Sep 05 06:38:44.828987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:44.828953    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb" event={"ID":"e50f8853-6b18-4c60-94df-a1769d459c2b","Type":"ContainerStarted","Data":"4e09bde10cfaff8bbbc329f0fda0371e6225ddf45523d4f147a53064cfd7a6bc"}
Sep 05 06:38:45.835777 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:45.835747    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb" event={"ID":"e50f8853-6b18-4c60-94df-a1769d459c2b","Type":"ContainerStarted","Data":"8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb"}
Sep 05 06:38:45.836195 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:45.835846    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:45.858096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:45.858050    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb" podStartSLOduration=3.484802785 podStartE2EDuration="3.858035823s" totalImagesPullingTime="373.233038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:38:45.855628834 +0000 UTC m=+774.223676042" watchObservedRunningTime="2026-09-05 06:38:45.858035823 +0000 UTC m=+774.226083032"
Sep 05 06:38:46.703058 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:46.703020    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:46.803080 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:46.803046    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:47.106932 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:47.106823    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:47.699915 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:47.699877    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:47.832177 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:47.832134    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:47.928203 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:47.928165    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:48.025096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:48.025065    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:49.146339 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:49.146305    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:49.834576 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:49.834544    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:38:51.728123 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:51.728090    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:51.846561 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:51.846528    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:38:53.278976 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:53.278934    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:53.340560 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:53.340529    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:53.477061 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:53.477008    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:53.570655 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:53.570555    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:53.825076 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:53.825006    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:38:53.866375 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:53.866341    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:54.120167 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:54.120079    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:54.831169 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:54.831141    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:38:54.831595 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:54.831375    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:38:57.128510 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:57.128458    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:57.714270 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:57.714235    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:58.141222 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:58.141157    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:38:58.481989 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:38:58.481887    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:00.109650 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:00.109610    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:00.211742 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:00.211708    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:01.179234 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:01.179198    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:01.858315 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:01.858279    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:01.956184 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:01.956141    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:02.346627 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:02.346596    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:03.317718 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.317680    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"]
Sep 05 06:39:03.322703 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.322685    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.325553 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.325520    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-7080a4-serving-cert\""
Sep 05 06:39:03.332840 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.332810    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"]
Sep 05 06:39:03.352169 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.352142    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:03.450503 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.450472    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:03.460640 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.460614    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-maas-api-tls\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.460741 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.460646    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26f67427-fe22-4937-b82a-5bcd687049db-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.460741 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.460729    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qc9x\" (UniqueName: \"kubernetes.io/projected/26f67427-fe22-4937-b82a-5bcd687049db-kube-api-access-2qc9x\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.460805 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.460756    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-metrics-certs\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.562010 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.561976    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-maas-api-tls\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.562010 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.562011    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26f67427-fe22-4937-b82a-5bcd687049db-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.562234 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.562078    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qc9x\" (UniqueName: \"kubernetes.io/projected/26f67427-fe22-4937-b82a-5bcd687049db-kube-api-access-2qc9x\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.562234 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.562116    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-metrics-certs\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.562234 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:39:03.562160    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-serving-cert: secret "maas-api-e2e-inf-b-7080a4-serving-cert" not found
Sep 05 06:39:03.562234 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:39:03.562231    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-maas-api-tls podName:26f67427-fe22-4937-b82a-5bcd687049db nodeName:}" failed. No retries permitted until 2026-09-05 06:39:04.062210214 +0000 UTC m=+792.430257422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-maas-api-tls") pod "maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66" (UID: "26f67427-fe22-4937-b82a-5bcd687049db") : secret "maas-api-e2e-inf-b-7080a4-serving-cert" not found
Sep 05 06:39:03.562742 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.562721    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26f67427-fe22-4937-b82a-5bcd687049db-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.564688 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.564662    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-metrics-certs\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.576192 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.574196    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qc9x\" (UniqueName: \"kubernetes.io/projected/26f67427-fe22-4937-b82a-5bcd687049db-kube-api-access-2qc9x\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:03.687885 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.687826    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:03.894768 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.894678    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:03.993608 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:03.993572    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:04.067377 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:04.067344    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-maas-api-tls\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:04.070146 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:04.070117    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-maas-api-tls\") pod \"maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:04.237060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:04.236980    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:04.367420 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:04.367389    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:04.509132 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:04.509108    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"]
Sep 05 06:39:04.934470 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:04.934440    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66" event={"ID":"26f67427-fe22-4937-b82a-5bcd687049db","Type":"ContainerStarted","Data":"d2d3d86c1f6949d60160902d7b33259b31626b04cb88019208e8104c501a0f59"}
Sep 05 06:39:05.454698 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:05.454666    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:05.940611 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:05.940572    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66" event={"ID":"26f67427-fe22-4937-b82a-5bcd687049db","Type":"ContainerStarted","Data":"ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61"}
Sep 05 06:39:05.940807 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:05.940693    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:05.961189 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:05.961130    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66" podStartSLOduration=2.599690551 podStartE2EDuration="2.96111325s" totalImagesPullingTime="361.422699ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:39:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:39:05.95977135 +0000 UTC m=+794.327818560" watchObservedRunningTime="2026-09-05 06:39:05.96111325 +0000 UTC m=+794.329160459"
Sep 05 06:39:06.227241 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:06.227146    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:07.128664 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:07.128630    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:08.112445 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:08.112411    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:10.674269 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:10.674231    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:11.951319 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:11.951287    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:39:21.644407 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:21.644310    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:22.340367 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.340333    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"]
Sep 05 06:39:22.340615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.340575    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm" podUID="074a3307-fbb5-48e2-9166-ec03ce6c3d88" containerName="maas-api" containerID="cri-o://44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed" gracePeriod=30
Sep 05 06:39:22.374279 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.374248    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:22.400347 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.400317    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"]
Sep 05 06:39:22.400564 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.400521    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5" podUID="649146e9-4424-4c5d-90df-cde530f16a71" containerName="payload-processing" containerID="cri-o://eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea" gracePeriod=30
Sep 05 06:39:22.406593 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.406567    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"]
Sep 05 06:39:22.406797 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.406772    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf" podUID="bf443cfb-2bfc-468d-a9b1-efff2780b1d4" containerName="payload-pre-processing" containerID="cri-o://3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c" gracePeriod=30
Sep 05 06:39:22.474085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.474053    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:22.575194 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.574785    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:22.637560 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.637425    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:22.656416 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.656392    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:39:22.680114 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.680087    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:39:22.715319 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.715290    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:39:22.734018 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.733931    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-maas-api-tls\") pod \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") "
Sep 05 06:39:22.734204 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.734117    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pznjg\" (UniqueName: \"kubernetes.io/projected/649146e9-4424-4c5d-90df-cde530f16a71-kube-api-access-pznjg\") pod \"649146e9-4424-4c5d-90df-cde530f16a71\" (UID: \"649146e9-4424-4c5d-90df-cde530f16a71\") "
Sep 05 06:39:22.734204 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.734181    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwhvr\" (UniqueName: \"kubernetes.io/projected/074a3307-fbb5-48e2-9166-ec03ce6c3d88-kube-api-access-wwhvr\") pod \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") "
Sep 05 06:39:22.734325 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.734207    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-metrics-certs\") pod \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") "
Sep 05 06:39:22.734325 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.734233    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-plugins-config-volume\") pod \"bf443cfb-2bfc-468d-a9b1-efff2780b1d4\" (UID: \"bf443cfb-2bfc-468d-a9b1-efff2780b1d4\") "
Sep 05 06:39:22.734325 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.734291    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcw6d\" (UniqueName: \"kubernetes.io/projected/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-kube-api-access-tcw6d\") pod \"bf443cfb-2bfc-468d-a9b1-efff2780b1d4\" (UID: \"bf443cfb-2bfc-468d-a9b1-efff2780b1d4\") "
Sep 05 06:39:22.734547 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.734325    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/649146e9-4424-4c5d-90df-cde530f16a71-plugins-config-volume\") pod \"649146e9-4424-4c5d-90df-cde530f16a71\" (UID: \"649146e9-4424-4c5d-90df-cde530f16a71\") "
Sep 05 06:39:22.734547 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.734405    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/074a3307-fbb5-48e2-9166-ec03ce6c3d88-openshift-service-ca\") pod \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\" (UID: \"074a3307-fbb5-48e2-9166-ec03ce6c3d88\") "
Sep 05 06:39:22.735580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.735258    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/074a3307-fbb5-48e2-9166-ec03ce6c3d88-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "074a3307-fbb5-48e2-9166-ec03ce6c3d88" (UID: "074a3307-fbb5-48e2-9166-ec03ce6c3d88"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:39:22.735580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.735402    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/649146e9-4424-4c5d-90df-cde530f16a71-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "649146e9-4424-4c5d-90df-cde530f16a71" (UID: "649146e9-4424-4c5d-90df-cde530f16a71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:39:22.735580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.735552    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf443cfb-2bfc-468d-a9b1-efff2780b1d4" (UID: "bf443cfb-2bfc-468d-a9b1-efff2780b1d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:39:22.739026 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.738562    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/649146e9-4424-4c5d-90df-cde530f16a71-kube-api-access-pznjg" (OuterVolumeSpecName: "kube-api-access-pznjg") pod "649146e9-4424-4c5d-90df-cde530f16a71" (UID: "649146e9-4424-4c5d-90df-cde530f16a71"). InnerVolumeSpecName "kube-api-access-pznjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:39:22.739026 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.738879    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "074a3307-fbb5-48e2-9166-ec03ce6c3d88" (UID: "074a3307-fbb5-48e2-9166-ec03ce6c3d88"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:39:22.739678 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.739639    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "074a3307-fbb5-48e2-9166-ec03ce6c3d88" (UID: "074a3307-fbb5-48e2-9166-ec03ce6c3d88"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:39:22.740198 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.740170    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/074a3307-fbb5-48e2-9166-ec03ce6c3d88-kube-api-access-wwhvr" (OuterVolumeSpecName: "kube-api-access-wwhvr") pod "074a3307-fbb5-48e2-9166-ec03ce6c3d88" (UID: "074a3307-fbb5-48e2-9166-ec03ce6c3d88"). InnerVolumeSpecName "kube-api-access-wwhvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:39:22.740535 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.740507    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-kube-api-access-tcw6d" (OuterVolumeSpecName: "kube-api-access-tcw6d") pod "bf443cfb-2bfc-468d-a9b1-efff2780b1d4" (UID: "bf443cfb-2bfc-468d-a9b1-efff2780b1d4"). InnerVolumeSpecName "kube-api-access-tcw6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:39:22.757287 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.757252    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:22.836139 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.836107    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/649146e9-4424-4c5d-90df-cde530f16a71-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:39:22.836139 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.836133    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/074a3307-fbb5-48e2-9166-ec03ce6c3d88-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:39:22.836139 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.836143    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:39:22.836367 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.836153    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pznjg\" (UniqueName: \"kubernetes.io/projected/649146e9-4424-4c5d-90df-cde530f16a71-kube-api-access-pznjg\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:39:22.836367 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.836161    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwhvr\" (UniqueName: \"kubernetes.io/projected/074a3307-fbb5-48e2-9166-ec03ce6c3d88-kube-api-access-wwhvr\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:39:22.836367 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.836169    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/074a3307-fbb5-48e2-9166-ec03ce6c3d88-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:39:22.836367 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.836177    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:39:22.836367 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:22.836185    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcw6d\" (UniqueName: \"kubernetes.io/projected/bf443cfb-2bfc-468d-a9b1-efff2780b1d4-kube-api-access-tcw6d\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:39:23.023981 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.023947    2573 generic.go:356] "Generic (PLEG): container finished" podID="bf443cfb-2bfc-468d-a9b1-efff2780b1d4" containerID="3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c" exitCode=0
Sep 05 06:39:23.024159 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.023993    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"
Sep 05 06:39:23.024159 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.024035    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf" event={"ID":"bf443cfb-2bfc-468d-a9b1-efff2780b1d4","Type":"ContainerDied","Data":"3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c"}
Sep 05 06:39:23.024159 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.024115    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf" event={"ID":"bf443cfb-2bfc-468d-a9b1-efff2780b1d4","Type":"ContainerDied","Data":"44e55b57cc221ce8a3bb068821835abee93166df50870015633818cab043ece3"}
Sep 05 06:39:23.024159 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.024140    2573 scope.go:117] "RemoveContainer" containerID="3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c"
Sep 05 06:39:23.025438 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.025418    2573 generic.go:356] "Generic (PLEG): container finished" podID="074a3307-fbb5-48e2-9166-ec03ce6c3d88" containerID="44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed" exitCode=0
Sep 05 06:39:23.025521 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.025480    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"
Sep 05 06:39:23.025521 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.025500    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm" event={"ID":"074a3307-fbb5-48e2-9166-ec03ce6c3d88","Type":"ContainerDied","Data":"44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed"}
Sep 05 06:39:23.025617 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.025553    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm" event={"ID":"074a3307-fbb5-48e2-9166-ec03ce6c3d88","Type":"ContainerDied","Data":"00bc001227eebf93d8dd9bcec80b7a0eb5cfbc0803c56c5360bc15262dbfd902"}
Sep 05 06:39:23.026998 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.026974    2573 generic.go:356] "Generic (PLEG): container finished" podID="649146e9-4424-4c5d-90df-cde530f16a71" containerID="eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea" exitCode=0
Sep 05 06:39:23.027107 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.027021    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"
Sep 05 06:39:23.027107 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.027032    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5" event={"ID":"649146e9-4424-4c5d-90df-cde530f16a71","Type":"ContainerDied","Data":"eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea"}
Sep 05 06:39:23.027107 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.027075    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5" event={"ID":"649146e9-4424-4c5d-90df-cde530f16a71","Type":"ContainerDied","Data":"a643cb1305b37a28b8a7b3f24645d8efee85bbff9d95b9a9fe746ebce33e19fe"}
Sep 05 06:39:23.037338 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.037320    2573 scope.go:117] "RemoveContainer" containerID="3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c"
Sep 05 06:39:23.037673 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:39:23.037653    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c\": container with ID starting with 3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c not found: ID does not exist" containerID="3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c"
Sep 05 06:39:23.037802 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.037685    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c"} err="failed to get container status \"3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c\": rpc error: code = NotFound desc = could not find container \"3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c\": container with ID starting with 3b3bf38110079cbb772d2478c65cbfd9498a8a53144cf3534369779703c7053c not found: ID does not exist"
Sep 05 06:39:23.037802 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.037745    2573 scope.go:117] "RemoveContainer" containerID="44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed"
Sep 05 06:39:23.046206 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.046190    2573 scope.go:117] "RemoveContainer" containerID="44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed"
Sep 05 06:39:23.046461 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:39:23.046443    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed\": container with ID starting with 44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed not found: ID does not exist" containerID="44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed"
Sep 05 06:39:23.046535 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.046472    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed"} err="failed to get container status \"44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed\": rpc error: code = NotFound desc = could not find container \"44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed\": container with ID starting with 44ac07b76dca6c30acc0cf54147964998e9db898b8a9013af6a5afee54d3faed not found: ID does not exist"
Sep 05 06:39:23.046535 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.046495    2573 scope.go:117] "RemoveContainer" containerID="eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea"
Sep 05 06:39:23.053492 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.053471    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"]
Sep 05 06:39:23.054122 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.054105    2573 scope.go:117] "RemoveContainer" containerID="eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea"
Sep 05 06:39:23.054381 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:39:23.054364    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea\": container with ID starting with eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea not found: ID does not exist" containerID="eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea"
Sep 05 06:39:23.054423 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.054387    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea"} err="failed to get container status \"eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea\": rpc error: code = NotFound desc = could not find container \"eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea\": container with ID starting with eee65081ad6bf8d7a7e4e992171b0114d62587bcfe03592f42c01b0fe35db1ea not found: ID does not exist"
Sep 05 06:39:23.058212 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.058190    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b1d1c87d-5d84c7c64b-zpsrf"]
Sep 05 06:39:23.069671 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.069647    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"]
Sep 05 06:39:23.078491 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.078470    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b1d1c87d-649b97d5dc-slcc5"]
Sep 05 06:39:23.093152 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.093130    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"]
Sep 05 06:39:23.096324 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.096306    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b1d1c87d-6676fbc7fb-vz8vm"]
Sep 05 06:39:23.306374 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.306284    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:23.397223 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.397188    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:23.505369 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.505330    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:23.888608 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.888577    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:23.985321 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:23.985281    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:24.155499 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:24.155424    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="074a3307-fbb5-48e2-9166-ec03ce6c3d88" path="/var/lib/kubelet/pods/074a3307-fbb5-48e2-9166-ec03ce6c3d88/volumes"
Sep 05 06:39:24.155783 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:24.155771    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="649146e9-4424-4c5d-90df-cde530f16a71" path="/var/lib/kubelet/pods/649146e9-4424-4c5d-90df-cde530f16a71/volumes"
Sep 05 06:39:24.156135 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:24.156121    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf443cfb-2bfc-468d-a9b1-efff2780b1d4" path="/var/lib/kubelet/pods/bf443cfb-2bfc-468d-a9b1-efff2780b1d4/volumes"
Sep 05 06:39:24.359989 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:24.359953    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:26.826182 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:26.826144    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:32.563135 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.563088    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:32.664408 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.664376    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:32.695511 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.695477    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"]
Sep 05 06:39:32.696107 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.696084    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf443cfb-2bfc-468d-a9b1-efff2780b1d4" containerName="payload-pre-processing"
Sep 05 06:39:32.696277 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.696264    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf443cfb-2bfc-468d-a9b1-efff2780b1d4" containerName="payload-pre-processing"
Sep 05 06:39:32.696382 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.696367    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="649146e9-4424-4c5d-90df-cde530f16a71" containerName="payload-processing"
Sep 05 06:39:32.696465 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.696405    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="649146e9-4424-4c5d-90df-cde530f16a71" containerName="payload-processing"
Sep 05 06:39:32.696465 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.696423    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="074a3307-fbb5-48e2-9166-ec03ce6c3d88" containerName="maas-api"
Sep 05 06:39:32.696465 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.696433    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="074a3307-fbb5-48e2-9166-ec03ce6c3d88" containerName="maas-api"
Sep 05 06:39:32.696608 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.696563    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="074a3307-fbb5-48e2-9166-ec03ce6c3d88" containerName="maas-api"
Sep 05 06:39:32.696608 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.696577    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="649146e9-4424-4c5d-90df-cde530f16a71" containerName="payload-processing"
Sep 05 06:39:32.696608 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.696590    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf443cfb-2bfc-468d-a9b1-efff2780b1d4" containerName="payload-pre-processing"
Sep 05 06:39:32.699880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.699862    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.702522 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.702499    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-2da41f75-serving-cert\""
Sep 05 06:39:32.712540 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.712511    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"]
Sep 05 06:39:32.723928 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.723904    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdchg\" (UniqueName: \"kubernetes.io/projected/9792262a-e96e-4d73-88b0-16e40cc654c6-kube-api-access-cdchg\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.724056 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.723938    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9792262a-e96e-4d73-88b0-16e40cc654c6-openshift-service-ca\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.724056 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.723964    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-metrics-certs\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.724130 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.724080    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-maas-api-tls\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.763559 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.763526    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:32.825395 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.825318    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdchg\" (UniqueName: \"kubernetes.io/projected/9792262a-e96e-4d73-88b0-16e40cc654c6-kube-api-access-cdchg\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.825395 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.825392    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9792262a-e96e-4d73-88b0-16e40cc654c6-openshift-service-ca\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.825649 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.825486    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-metrics-certs\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.825976 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.825949    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-maas-api-tls\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.826108 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:39:32.826091    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-serving-cert: secret "maas-api-e2e-migrate-2da41f75-serving-cert" not found
Sep 05 06:39:32.826172 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.826116    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9792262a-e96e-4d73-88b0-16e40cc654c6-openshift-service-ca\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.826172 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:39:32.826169    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-maas-api-tls podName:9792262a-e96e-4d73-88b0-16e40cc654c6 nodeName:}" failed. No retries permitted until 2026-09-05 06:39:33.326149075 +0000 UTC m=+821.694196267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-maas-api-tls") pod "maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f" (UID: "9792262a-e96e-4d73-88b0-16e40cc654c6") : secret "maas-api-e2e-migrate-2da41f75-serving-cert" not found
Sep 05 06:39:32.827933 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.827913    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-metrics-certs\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.837252 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.837224    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdchg\" (UniqueName: \"kubernetes.io/projected/9792262a-e96e-4d73-88b0-16e40cc654c6-kube-api-access-cdchg\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:32.864179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.864142    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:32.967671 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:32.967636    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:33.330693 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.330661    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-maas-api-tls\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:33.333093 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.333070    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-maas-api-tls\") pod \"maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:33.363155 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.363118    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:33.460391 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.460356    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:33.480923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.480893    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"]
Sep 05 06:39:33.484542 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.484522    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:39:33.486895 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.486871    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-2da41f75-dockercfg-cmnrk\""
Sep 05 06:39:33.486991 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.486940    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-2da41f75\""
Sep 05 06:39:33.495823 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.495801    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"]
Sep 05 06:39:33.533057 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.533023    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3184ef68-e255-436d-8d49-01fe2601e8b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx\" (UID: \"3184ef68-e255-436d-8d49-01fe2601e8b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:39:33.533219 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.533169    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfb2n\" (UniqueName: \"kubernetes.io/projected/3184ef68-e255-436d-8d49-01fe2601e8b2-kube-api-access-hfb2n\") pod \"payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx\" (UID: \"3184ef68-e255-436d-8d49-01fe2601e8b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:39:33.612464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.612385    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:33.634453 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.634423    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfb2n\" (UniqueName: \"kubernetes.io/projected/3184ef68-e255-436d-8d49-01fe2601e8b2-kube-api-access-hfb2n\") pod \"payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx\" (UID: \"3184ef68-e255-436d-8d49-01fe2601e8b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:39:33.634453 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.634467    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3184ef68-e255-436d-8d49-01fe2601e8b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx\" (UID: \"3184ef68-e255-436d-8d49-01fe2601e8b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:39:33.635188 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.635167    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3184ef68-e255-436d-8d49-01fe2601e8b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx\" (UID: \"3184ef68-e255-436d-8d49-01fe2601e8b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:39:33.648315 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.648283    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfb2n\" (UniqueName: \"kubernetes.io/projected/3184ef68-e255-436d-8d49-01fe2601e8b2-kube-api-access-hfb2n\") pod \"payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx\" (UID: \"3184ef68-e255-436d-8d49-01fe2601e8b2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:39:33.795656 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.795622    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:39:33.885290 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:33.885268    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"]
Sep 05 06:39:33.887567 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:39:33.887523    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9792262a_e96e_4d73_88b0_16e40cc654c6.slice/crio-8c6fa6f5209b30c172d36a5eb9016016834950a9836a788907967f749f167b49 WatchSource:0}: Error finding container 8c6fa6f5209b30c172d36a5eb9016016834950a9836a788907967f749f167b49: Status 404 returned error can't find the container with id 8c6fa6f5209b30c172d36a5eb9016016834950a9836a788907967f749f167b49
Sep 05 06:39:34.069262 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:34.069237    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"]
Sep 05 06:39:34.071127 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:39:34.071101    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3184ef68_e255_436d_8d49_01fe2601e8b2.slice/crio-c44de13528002710674d45b7485bcd9c0f6c0a643e635b2c33e1ca428c6d9d45 WatchSource:0}: Error finding container c44de13528002710674d45b7485bcd9c0f6c0a643e635b2c33e1ca428c6d9d45: Status 404 returned error can't find the container with id c44de13528002710674d45b7485bcd9c0f6c0a643e635b2c33e1ca428c6d9d45
Sep 05 06:39:34.085328 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:34.085302    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f" event={"ID":"9792262a-e96e-4d73-88b0-16e40cc654c6","Type":"ContainerStarted","Data":"8c6fa6f5209b30c172d36a5eb9016016834950a9836a788907967f749f167b49"}
Sep 05 06:39:34.086583 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:34.086557    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx" event={"ID":"3184ef68-e255-436d-8d49-01fe2601e8b2","Type":"ContainerStarted","Data":"c44de13528002710674d45b7485bcd9c0f6c0a643e635b2c33e1ca428c6d9d45"}
Sep 05 06:39:35.092982 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:35.092940    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f" event={"ID":"9792262a-e96e-4d73-88b0-16e40cc654c6","Type":"ContainerStarted","Data":"6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56"}
Sep 05 06:39:35.093429 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:35.093083    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:35.094532 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:35.094509    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx" event={"ID":"3184ef68-e255-436d-8d49-01fe2601e8b2","Type":"ContainerStarted","Data":"98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18"}
Sep 05 06:39:35.094648 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:35.094559    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:39:35.115045 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:35.115005    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f" podStartSLOduration=2.804486443 podStartE2EDuration="3.114990779s" totalImagesPullingTime="310.504336ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:39:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:39:35.114253443 +0000 UTC m=+823.482300654" watchObservedRunningTime="2026-09-05 06:39:35.114990779 +0000 UTC m=+823.483037989"
Sep 05 06:39:35.133836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:35.133792    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx" podStartSLOduration=2.133780028 podStartE2EDuration="2.133780028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:39:35.131688712 +0000 UTC m=+823.499735947" watchObservedRunningTime="2026-09-05 06:39:35.133780028 +0000 UTC m=+823.501827238"
Sep 05 06:39:38.978427 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:38.978393    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"]
Sep 05 06:39:38.983260 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:38.983239    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:38.985716 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:38.985697    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-c4e74764-kserve-self-signed-certs\""
Sep 05 06:39:38.992496 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:38.992473    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"]
Sep 05 06:39:39.022336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.022306    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:39:39.086791 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.086755    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-home\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.086980 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.086936    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6nx7\" (UniqueName: \"kubernetes.io/projected/bb94912d-19f3-4a54-8ec0-8c427a796550-kube-api-access-s6nx7\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.087044 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.086993    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb94912d-19f3-4a54-8ec0-8c427a796550-tls-certs\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.087129 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.087067    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-dshm\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.087129 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.087090    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-model-cache\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.087129 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.087112    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-tmp-dir\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.188258 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.188226    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6nx7\" (UniqueName: \"kubernetes.io/projected/bb94912d-19f3-4a54-8ec0-8c427a796550-kube-api-access-s6nx7\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.188258 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.188267    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb94912d-19f3-4a54-8ec0-8c427a796550-tls-certs\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.188464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.188378    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-dshm\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.188464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.188396    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-model-cache\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.188464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.188411    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-tmp-dir\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.188464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.188456    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-home\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.188827 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.188810    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-home\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.188932 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.188889    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-model-cache\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.188990 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.188925    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-tmp-dir\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.190550 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.190527    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-dshm\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.190937 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.190917    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb94912d-19f3-4a54-8ec0-8c427a796550-tls-certs\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.196793 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.196774    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6nx7\" (UniqueName: \"kubernetes.io/projected/bb94912d-19f3-4a54-8ec0-8c427a796550-kube-api-access-s6nx7\") pod \"e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") " pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.294105 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.294070    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:39.564842 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:39.564819    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"]
Sep 05 06:39:39.567152 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:39:39.567126    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb94912d_19f3_4a54_8ec0_8c427a796550.slice/crio-c87ae68d1b123d6f05a4b715e026ab658a763905cd666089595109eefd256f6d WatchSource:0}: Error finding container c87ae68d1b123d6f05a4b715e026ab658a763905cd666089595109eefd256f6d: Status 404 returned error can't find the container with id c87ae68d1b123d6f05a4b715e026ab658a763905cd666089595109eefd256f6d
Sep 05 06:39:40.121271 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:40.121224    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl" event={"ID":"bb94912d-19f3-4a54-8ec0-8c427a796550","Type":"ContainerStarted","Data":"746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c"}
Sep 05 06:39:40.121643 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:40.121280    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl" event={"ID":"bb94912d-19f3-4a54-8ec0-8c427a796550","Type":"ContainerStarted","Data":"c87ae68d1b123d6f05a4b715e026ab658a763905cd666089595109eefd256f6d"}
Sep 05 06:39:40.142552 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:40.142497    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl" podStartSLOduration=2.142475016 podStartE2EDuration="2.142475016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:39:40.142098865 +0000 UTC m=+828.510146086" watchObservedRunningTime="2026-09-05 06:39:40.142475016 +0000 UTC m=+828.510522228"
Sep 05 06:39:41.105583 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:41.105551    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:39:46.101583 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:46.101545    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:39:49.295193 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:49.295156    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:49.295193 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:49.295205    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:49.308312 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:49.308284    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:39:50.188966 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:39:50.188930    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:40:00.391826 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:00.391789    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:04.137639 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.137606    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"]
Sep 05 06:40:04.138052 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.137865    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f" podUID="9792262a-e96e-4d73-88b0-16e40cc654c6" containerName="maas-api" containerID="cri-o://6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56" gracePeriod=30
Sep 05 06:40:04.170266 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.170229    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:04.208041 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.208005    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"]
Sep 05 06:40:04.208310 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.208280    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx" podUID="3184ef68-e255-436d-8d49-01fe2601e8b2" containerName="payload-pre-processing" containerID="cri-o://98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18" gracePeriod=30
Sep 05 06:40:04.270188 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.270160    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:04.452172 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.452143    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:40:04.488542 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.488519    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:40:04.510161 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.510121    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:04.536530 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.536499    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-metrics-certs\") pod \"9792262a-e96e-4d73-88b0-16e40cc654c6\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") "
Sep 05 06:40:04.536684 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.536549    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9792262a-e96e-4d73-88b0-16e40cc654c6-openshift-service-ca\") pod \"9792262a-e96e-4d73-88b0-16e40cc654c6\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") "
Sep 05 06:40:04.536684 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.536619    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdchg\" (UniqueName: \"kubernetes.io/projected/9792262a-e96e-4d73-88b0-16e40cc654c6-kube-api-access-cdchg\") pod \"9792262a-e96e-4d73-88b0-16e40cc654c6\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") "
Sep 05 06:40:04.536684 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.536655    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3184ef68-e255-436d-8d49-01fe2601e8b2-plugins-config-volume\") pod \"3184ef68-e255-436d-8d49-01fe2601e8b2\" (UID: \"3184ef68-e255-436d-8d49-01fe2601e8b2\") "
Sep 05 06:40:04.536884 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.536684    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-maas-api-tls\") pod \"9792262a-e96e-4d73-88b0-16e40cc654c6\" (UID: \"9792262a-e96e-4d73-88b0-16e40cc654c6\") "
Sep 05 06:40:04.536884 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.536733    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfb2n\" (UniqueName: \"kubernetes.io/projected/3184ef68-e255-436d-8d49-01fe2601e8b2-kube-api-access-hfb2n\") pod \"3184ef68-e255-436d-8d49-01fe2601e8b2\" (UID: \"3184ef68-e255-436d-8d49-01fe2601e8b2\") "
Sep 05 06:40:04.537004 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.536956    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9792262a-e96e-4d73-88b0-16e40cc654c6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9792262a-e96e-4d73-88b0-16e40cc654c6" (UID: "9792262a-e96e-4d73-88b0-16e40cc654c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:40:04.537064 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.537049    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9792262a-e96e-4d73-88b0-16e40cc654c6-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:04.537121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.537078    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3184ef68-e255-436d-8d49-01fe2601e8b2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3184ef68-e255-436d-8d49-01fe2601e8b2" (UID: "3184ef68-e255-436d-8d49-01fe2601e8b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:40:04.539085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.539042    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9792262a-e96e-4d73-88b0-16e40cc654c6-kube-api-access-cdchg" (OuterVolumeSpecName: "kube-api-access-cdchg") pod "9792262a-e96e-4d73-88b0-16e40cc654c6" (UID: "9792262a-e96e-4d73-88b0-16e40cc654c6"). InnerVolumeSpecName "kube-api-access-cdchg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:04.539191 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.539083    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9792262a-e96e-4d73-88b0-16e40cc654c6" (UID: "9792262a-e96e-4d73-88b0-16e40cc654c6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:40:04.539424 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.539395    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9792262a-e96e-4d73-88b0-16e40cc654c6" (UID: "9792262a-e96e-4d73-88b0-16e40cc654c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:40:04.539513 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.539401    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3184ef68-e255-436d-8d49-01fe2601e8b2-kube-api-access-hfb2n" (OuterVolumeSpecName: "kube-api-access-hfb2n") pod "3184ef68-e255-436d-8d49-01fe2601e8b2" (UID: "3184ef68-e255-436d-8d49-01fe2601e8b2"). InnerVolumeSpecName "kube-api-access-hfb2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:04.605090 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.605058    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:04.637632 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.637602    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:04.637632 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.637630    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdchg\" (UniqueName: \"kubernetes.io/projected/9792262a-e96e-4d73-88b0-16e40cc654c6-kube-api-access-cdchg\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:04.637827 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.637644    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3184ef68-e255-436d-8d49-01fe2601e8b2-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:04.637827 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.637658    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9792262a-e96e-4d73-88b0-16e40cc654c6-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:04.637827 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:04.637670    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfb2n\" (UniqueName: \"kubernetes.io/projected/3184ef68-e255-436d-8d49-01fe2601e8b2-kube-api-access-hfb2n\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:05.032396 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.032359    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:05.254722 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.254688    2573 generic.go:356] "Generic (PLEG): container finished" podID="9792262a-e96e-4d73-88b0-16e40cc654c6" containerID="6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56" exitCode=0
Sep 05 06:40:05.254722 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.254734    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"
Sep 05 06:40:05.255268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.254763    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f" event={"ID":"9792262a-e96e-4d73-88b0-16e40cc654c6","Type":"ContainerDied","Data":"6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56"}
Sep 05 06:40:05.255268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.254821    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f" event={"ID":"9792262a-e96e-4d73-88b0-16e40cc654c6","Type":"ContainerDied","Data":"8c6fa6f5209b30c172d36a5eb9016016834950a9836a788907967f749f167b49"}
Sep 05 06:40:05.255268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.254881    2573 scope.go:117] "RemoveContainer" containerID="6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56"
Sep 05 06:40:05.256379 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.256301    2573 generic.go:356] "Generic (PLEG): container finished" podID="3184ef68-e255-436d-8d49-01fe2601e8b2" containerID="98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18" exitCode=0
Sep 05 06:40:05.256379 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.256342    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"
Sep 05 06:40:05.256482 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.256386    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx" event={"ID":"3184ef68-e255-436d-8d49-01fe2601e8b2","Type":"ContainerDied","Data":"98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18"}
Sep 05 06:40:05.256482 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.256449    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx" event={"ID":"3184ef68-e255-436d-8d49-01fe2601e8b2","Type":"ContainerDied","Data":"c44de13528002710674d45b7485bcd9c0f6c0a643e635b2c33e1ca428c6d9d45"}
Sep 05 06:40:05.264802 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.264785    2573 scope.go:117] "RemoveContainer" containerID="6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56"
Sep 05 06:40:05.265197 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:40:05.265164    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56\": container with ID starting with 6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56 not found: ID does not exist" containerID="6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56"
Sep 05 06:40:05.265297 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.265201    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56"} err="failed to get container status \"6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56\": rpc error: code = NotFound desc = could not find container \"6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56\": container with ID starting with 6b7854758801ce30892b9719ae52cd02d177deff06a09042e0a1e4a3ebfadc56 not found: ID does not exist"
Sep 05 06:40:05.265297 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.265220    2573 scope.go:117] "RemoveContainer" containerID="98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18"
Sep 05 06:40:05.274038 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.274022    2573 scope.go:117] "RemoveContainer" containerID="98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18"
Sep 05 06:40:05.274310 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:40:05.274292    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18\": container with ID starting with 98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18 not found: ID does not exist" containerID="98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18"
Sep 05 06:40:05.274374 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.274315    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18"} err="failed to get container status \"98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18\": rpc error: code = NotFound desc = could not find container \"98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18\": container with ID starting with 98bc8af01a0ebda34c939fdc75b9f1b6f29b73e48fd911ea34bbd17499b53c18 not found: ID does not exist"
Sep 05 06:40:05.279922 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.279902    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"]
Sep 05 06:40:05.286165 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.286111    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2da41f75-6b6946cd55-8754f"]
Sep 05 06:40:05.302827 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.302798    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"]
Sep 05 06:40:05.317836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.317806    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2da41f75-5dc6f8dc5d-rfzfx"]
Sep 05 06:40:05.361837 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.361797    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:05.506296 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:05.506262    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:06.156756 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:06.156725    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3184ef68-e255-436d-8d49-01fe2601e8b2" path="/var/lib/kubelet/pods/3184ef68-e255-436d-8d49-01fe2601e8b2/volumes"
Sep 05 06:40:06.157197 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:06.157181    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9792262a-e96e-4d73-88b0-16e40cc654c6" path="/var/lib/kubelet/pods/9792262a-e96e-4d73-88b0-16e40cc654c6/volumes"
Sep 05 06:40:07.725293 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:07.725254    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:07.825285 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:07.825248    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:09.002724 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.002687    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"]
Sep 05 06:40:09.003335 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.003307    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9792262a-e96e-4d73-88b0-16e40cc654c6" containerName="maas-api"
Sep 05 06:40:09.003335 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.003331    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9792262a-e96e-4d73-88b0-16e40cc654c6" containerName="maas-api"
Sep 05 06:40:09.003509 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.003353    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3184ef68-e255-436d-8d49-01fe2601e8b2" containerName="payload-pre-processing"
Sep 05 06:40:09.003509 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.003363    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3184ef68-e255-436d-8d49-01fe2601e8b2" containerName="payload-pre-processing"
Sep 05 06:40:09.003509 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.003473    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9792262a-e96e-4d73-88b0-16e40cc654c6" containerName="maas-api"
Sep 05 06:40:09.003509 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.003493    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3184ef68-e255-436d-8d49-01fe2601e8b2" containerName="payload-pre-processing"
Sep 05 06:40:09.008453 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.008428    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.010863 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.010824    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bba7714e-serving-cert\""
Sep 05 06:40:09.017921 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.017844    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"]
Sep 05 06:40:09.050652 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.050618    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:09.079872 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.079808    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e4ba35-1735-4b33-b2a6-ac355463b75e-openshift-service-ca\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.080040 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.079902    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzxcz\" (UniqueName: \"kubernetes.io/projected/f5e4ba35-1735-4b33-b2a6-ac355463b75e-kube-api-access-wzxcz\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.080040 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.079941    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-metrics-certs\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.080040 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.079967    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-maas-api-tls\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.153471 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.153440    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:09.180755 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.180722    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e4ba35-1735-4b33-b2a6-ac355463b75e-openshift-service-ca\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.180997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.180797    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzxcz\" (UniqueName: \"kubernetes.io/projected/f5e4ba35-1735-4b33-b2a6-ac355463b75e-kube-api-access-wzxcz\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.180997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.180878    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-metrics-certs\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.180997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.180930    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-maas-api-tls\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.181186 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:40:09.181055    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-serving-cert: secret "maas-api-e2e-ait-bba7714e-serving-cert" not found
Sep 05 06:40:09.181186 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:40:09.181115    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-maas-api-tls podName:f5e4ba35-1735-4b33-b2a6-ac355463b75e nodeName:}" failed. No retries permitted until 2026-09-05 06:40:09.681095265 +0000 UTC m=+858.049142467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-maas-api-tls") pod "maas-api-e2e-ait-bba7714e-6848c698c9-gclzx" (UID: "f5e4ba35-1735-4b33-b2a6-ac355463b75e") : secret "maas-api-e2e-ait-bba7714e-serving-cert" not found
Sep 05 06:40:09.181529 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.181507    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e4ba35-1735-4b33-b2a6-ac355463b75e-openshift-service-ca\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.183457 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.183437    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-metrics-certs\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.189909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.189882    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzxcz\" (UniqueName: \"kubernetes.io/projected/f5e4ba35-1735-4b33-b2a6-ac355463b75e-kube-api-access-wzxcz\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.352236 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.352141    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:09.667422 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.667320    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:09.688570 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.688532    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-maas-api-tls\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.691022 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.690997    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-maas-api-tls\") pod \"maas-api-e2e-ait-bba7714e-6848c698c9-gclzx\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.763305 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.763258    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:09.836699 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.836660    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"]
Sep 05 06:40:09.840617 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.840594    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:09.843117 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.843094    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-bba7714e-dockercfg-8zbbd\""
Sep 05 06:40:09.843235 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.843125    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-bba7714e\""
Sep 05 06:40:09.851266 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.851241    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"]
Sep 05 06:40:09.890934 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.890896    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10368558-1f6e-47b7-b096-512ef0fe4291-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78\" (UID: \"10368558-1f6e-47b7-b096-512ef0fe4291\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:09.891099 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.890989    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27rrd\" (UniqueName: \"kubernetes.io/projected/10368558-1f6e-47b7-b096-512ef0fe4291-kube-api-access-27rrd\") pod \"payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78\" (UID: \"10368558-1f6e-47b7-b096-512ef0fe4291\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:09.923323 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.923248    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:09.991672 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.991640    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10368558-1f6e-47b7-b096-512ef0fe4291-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78\" (UID: \"10368558-1f6e-47b7-b096-512ef0fe4291\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:09.991921 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.991894    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27rrd\" (UniqueName: \"kubernetes.io/projected/10368558-1f6e-47b7-b096-512ef0fe4291-kube-api-access-27rrd\") pod \"payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78\" (UID: \"10368558-1f6e-47b7-b096-512ef0fe4291\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:09.992595 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:09.992571    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10368558-1f6e-47b7-b096-512ef0fe4291-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78\" (UID: \"10368558-1f6e-47b7-b096-512ef0fe4291\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:10.002651 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:10.002626    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27rrd\" (UniqueName: \"kubernetes.io/projected/10368558-1f6e-47b7-b096-512ef0fe4291-kube-api-access-27rrd\") pod \"payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78\" (UID: \"10368558-1f6e-47b7-b096-512ef0fe4291\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:10.153721 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:10.153691    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:10.198188 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:10.198026    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"]
Sep 05 06:40:10.200043 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:40:10.200016    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5e4ba35_1735_4b33_b2a6_ac355463b75e.slice/crio-ba53653daccf926eaa9060ad460c9ad9e962d76f9c070bf4571a7ddee3b63763 WatchSource:0}: Error finding container ba53653daccf926eaa9060ad460c9ad9e962d76f9c070bf4571a7ddee3b63763: Status 404 returned error can't find the container with id ba53653daccf926eaa9060ad460c9ad9e962d76f9c070bf4571a7ddee3b63763
Sep 05 06:40:10.286554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:10.286518    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx" event={"ID":"f5e4ba35-1735-4b33-b2a6-ac355463b75e","Type":"ContainerStarted","Data":"ba53653daccf926eaa9060ad460c9ad9e962d76f9c070bf4571a7ddee3b63763"}
Sep 05 06:40:10.433907 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:10.433883    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"]
Sep 05 06:40:10.435814 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:40:10.435783    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10368558_1f6e_47b7_b096_512ef0fe4291.slice/crio-266e8bd58d996dd2076ad5cee0fef13bc6c62bc39c22ff407ae82f8cc362ec3e WatchSource:0}: Error finding container 266e8bd58d996dd2076ad5cee0fef13bc6c62bc39c22ff407ae82f8cc362ec3e: Status 404 returned error can't find the container with id 266e8bd58d996dd2076ad5cee0fef13bc6c62bc39c22ff407ae82f8cc362ec3e
Sep 05 06:40:11.293335 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:11.293292    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx" event={"ID":"f5e4ba35-1735-4b33-b2a6-ac355463b75e","Type":"ContainerStarted","Data":"975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266"}
Sep 05 06:40:11.293801 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:11.293358    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:11.294945 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:11.294920    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78" event={"ID":"10368558-1f6e-47b7-b096-512ef0fe4291","Type":"ContainerStarted","Data":"0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f"}
Sep 05 06:40:11.295005 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:11.294969    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78" event={"ID":"10368558-1f6e-47b7-b096-512ef0fe4291","Type":"ContainerStarted","Data":"266e8bd58d996dd2076ad5cee0fef13bc6c62bc39c22ff407ae82f8cc362ec3e"}
Sep 05 06:40:11.295049 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:11.295037    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:11.318067 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:11.318015    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx" podStartSLOduration=2.940317113 podStartE2EDuration="3.317999125s" totalImagesPullingTime="377.682012ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:11.316172119 +0000 UTC m=+859.684219340" watchObservedRunningTime="2026-09-05 06:40:11.317999125 +0000 UTC m=+859.686046335"
Sep 05 06:40:11.337716 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:11.337660    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78" podStartSLOduration=2.337639326 podStartE2EDuration="2.337639326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:11.335937061 +0000 UTC m=+859.703984271" watchObservedRunningTime="2026-09-05 06:40:11.337639326 +0000 UTC m=+859.705686580"
Sep 05 06:40:17.305827 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:17.305793    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:22.301913 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:22.301878    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:27.087251 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.087217    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"]
Sep 05 06:40:27.093363 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.093318    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.098464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.098432    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-45e697\"/\"test-model-45e697-kserve-self-signed-certs\""
Sep 05 06:40:27.098464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.098460    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-45e697\"/\"kube-root-ca.crt\""
Sep 05 06:40:27.098653 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.098501    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-45e697\"/\"default-dockercfg-fwc8d\""
Sep 05 06:40:27.098740 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.098720    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-45e697\"/\"openshift-service-ca.crt\""
Sep 05 06:40:27.102508 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.102480    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"]
Sep 05 06:40:27.122845 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.122799    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:27.160864 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.160827    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-home\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.161032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.160910    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-model-cache\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.161032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.160945    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tmp-dir\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.161032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.160975    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-dshm\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.161032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.161003    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfmf6\" (UniqueName: \"kubernetes.io/projected/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-kube-api-access-sfmf6\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.161173 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.161034    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tls-certs\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.262565 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.262529    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfmf6\" (UniqueName: \"kubernetes.io/projected/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-kube-api-access-sfmf6\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.262767 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.262579    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tls-certs\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.262767 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.262637    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-home\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.262767 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.262671    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-model-cache\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.262767 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.262710    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tmp-dir\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.263027 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.262823    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-dshm\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.263203 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.263182    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-home\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.263259 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.263229    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tmp-dir\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.263259 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.263242    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-model-cache\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.264950 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.264925    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-dshm\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.265179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.265161    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tls-certs\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.273176 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.273142    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfmf6\" (UniqueName: \"kubernetes.io/projected/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-kube-api-access-sfmf6\") pod \"test-model-45e697-kserve-545757cb4f-mrhrd\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") " pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.406441 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.406354    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:27.688035 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:27.688009    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"]
Sep 05 06:40:27.689718 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:40:27.689690    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc0ae0a7_ccdd_43b2_b841_efe4c144dcbf.slice/crio-604bbfa53ab8d7df1dc53e2435afc528874dd44e4a60f9ab5c711835de185b1c WatchSource:0}: Error finding container 604bbfa53ab8d7df1dc53e2435afc528874dd44e4a60f9ab5c711835de185b1c: Status 404 returned error can't find the container with id 604bbfa53ab8d7df1dc53e2435afc528874dd44e4a60f9ab5c711835de185b1c
Sep 05 06:40:28.379309 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:28.379274    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd" event={"ID":"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf","Type":"ContainerStarted","Data":"89358aa8b3d6d4ea9ec98e5bd6d421c492fc5e46edb043128ba7717b67885486"}
Sep 05 06:40:28.379309 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:28.379315    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd" event={"ID":"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf","Type":"ContainerStarted","Data":"604bbfa53ab8d7df1dc53e2435afc528874dd44e4a60f9ab5c711835de185b1c"}
Sep 05 06:40:28.399729 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:28.399682    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd" podStartSLOduration=1.399665565 podStartE2EDuration="1.399665565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:28.397330458 +0000 UTC m=+876.765377668" watchObservedRunningTime="2026-09-05 06:40:28.399665565 +0000 UTC m=+876.767712774"
Sep 05 06:40:28.967735 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:28.967700    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"]
Sep 05 06:40:28.995266 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:28.995235    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:30.620994 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:30.620955    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:31.114978 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:31.114944    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:31.403561 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:31.403469    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:32.211786 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:32.211726    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:37.342693 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:37.342651    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:37.404415 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:37.404378    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:37.407078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:37.407059    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:37.407350 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:37.407328    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:37.420576 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:37.420550    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:37.437006 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:37.436979    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:40:41.052269 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:41.052237    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:41.893680 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:41.893646    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"]
Sep 05 06:40:41.894061 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:41.894029    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb" podUID="e50f8853-6b18-4c60-94df-a1769d459c2b" containerName="maas-api" containerID="cri-o://8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb" gracePeriod=30
Sep 05 06:40:41.925757 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:41.925716    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:41.961758 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:41.961726    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"]
Sep 05 06:40:41.961997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:41.961965    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc" podUID="0593df1b-a65b-4445-ab71-4e71cc10bd00" containerName="payload-processing" containerID="cri-o://6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced" gracePeriod=30
Sep 05 06:40:42.025971 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.025938    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:42.125535 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.125495    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:42.199467 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.199436    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:42.202817 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.202796    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:40:42.204394 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.204372    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rc9p7\" (UniqueName: \"kubernetes.io/projected/e50f8853-6b18-4c60-94df-a1769d459c2b-kube-api-access-rc9p7\") pod \"e50f8853-6b18-4c60-94df-a1769d459c2b\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") "
Sep 05 06:40:42.204507 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.204412    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e50f8853-6b18-4c60-94df-a1769d459c2b-openshift-service-ca\") pod \"e50f8853-6b18-4c60-94df-a1769d459c2b\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") "
Sep 05 06:40:42.204507 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.204501    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-metrics-certs\") pod \"e50f8853-6b18-4c60-94df-a1769d459c2b\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") "
Sep 05 06:40:42.204594 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.204520    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls\") pod \"e50f8853-6b18-4c60-94df-a1769d459c2b\" (UID: \"e50f8853-6b18-4c60-94df-a1769d459c2b\") "
Sep 05 06:40:42.204845 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.204819    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e50f8853-6b18-4c60-94df-a1769d459c2b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e50f8853-6b18-4c60-94df-a1769d459c2b" (UID: "e50f8853-6b18-4c60-94df-a1769d459c2b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:40:42.207285 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.207254    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e50f8853-6b18-4c60-94df-a1769d459c2b-kube-api-access-rc9p7" (OuterVolumeSpecName: "kube-api-access-rc9p7") pod "e50f8853-6b18-4c60-94df-a1769d459c2b" (UID: "e50f8853-6b18-4c60-94df-a1769d459c2b"). InnerVolumeSpecName "kube-api-access-rc9p7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:42.207389 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.207289    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e50f8853-6b18-4c60-94df-a1769d459c2b" (UID: "e50f8853-6b18-4c60-94df-a1769d459c2b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:40:42.207389 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.207307    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e50f8853-6b18-4c60-94df-a1769d459c2b" (UID: "e50f8853-6b18-4c60-94df-a1769d459c2b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:40:42.235624 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.235602    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:40:42.305043 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.305016    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp7vl\" (UniqueName: \"kubernetes.io/projected/0593df1b-a65b-4445-ab71-4e71cc10bd00-kube-api-access-dp7vl\") pod \"0593df1b-a65b-4445-ab71-4e71cc10bd00\" (UID: \"0593df1b-a65b-4445-ab71-4e71cc10bd00\") "
Sep 05 06:40:42.305286 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.305114    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0593df1b-a65b-4445-ab71-4e71cc10bd00-plugins-config-volume\") pod \"0593df1b-a65b-4445-ab71-4e71cc10bd00\" (UID: \"0593df1b-a65b-4445-ab71-4e71cc10bd00\") "
Sep 05 06:40:42.305441 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.305422    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:42.305498 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.305445    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e50f8853-6b18-4c60-94df-a1769d459c2b-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:42.305498 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.305462    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc9p7\" (UniqueName: \"kubernetes.io/projected/e50f8853-6b18-4c60-94df-a1769d459c2b-kube-api-access-rc9p7\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:42.305498 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.305476    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e50f8853-6b18-4c60-94df-a1769d459c2b-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:42.305610 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.305509    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0593df1b-a65b-4445-ab71-4e71cc10bd00-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0593df1b-a65b-4445-ab71-4e71cc10bd00" (UID: "0593df1b-a65b-4445-ab71-4e71cc10bd00"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:40:42.307124 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.307102    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0593df1b-a65b-4445-ab71-4e71cc10bd00-kube-api-access-dp7vl" (OuterVolumeSpecName: "kube-api-access-dp7vl") pod "0593df1b-a65b-4445-ab71-4e71cc10bd00" (UID: "0593df1b-a65b-4445-ab71-4e71cc10bd00"). InnerVolumeSpecName "kube-api-access-dp7vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:42.406396 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.406344    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp7vl\" (UniqueName: \"kubernetes.io/projected/0593df1b-a65b-4445-ab71-4e71cc10bd00-kube-api-access-dp7vl\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:42.406396 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.406387    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0593df1b-a65b-4445-ab71-4e71cc10bd00-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:42.452276 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.452192    2573 generic.go:356] "Generic (PLEG): container finished" podID="0593df1b-a65b-4445-ab71-4e71cc10bd00" containerID="6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced" exitCode=0
Sep 05 06:40:42.452276 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.452250    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"
Sep 05 06:40:42.452505 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.452276    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc" event={"ID":"0593df1b-a65b-4445-ab71-4e71cc10bd00","Type":"ContainerDied","Data":"6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced"}
Sep 05 06:40:42.452505 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.452357    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc" event={"ID":"0593df1b-a65b-4445-ab71-4e71cc10bd00","Type":"ContainerDied","Data":"c5449f50a5b10737ddfba303b76fc703b693ab446a655e315b8831253a056a12"}
Sep 05 06:40:42.452505 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.452381    2573 scope.go:117] "RemoveContainer" containerID="6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced"
Sep 05 06:40:42.453908 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.453883    2573 generic.go:356] "Generic (PLEG): container finished" podID="e50f8853-6b18-4c60-94df-a1769d459c2b" containerID="8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb" exitCode=0
Sep 05 06:40:42.454007 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.453913    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb" event={"ID":"e50f8853-6b18-4c60-94df-a1769d459c2b","Type":"ContainerDied","Data":"8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb"}
Sep 05 06:40:42.454007 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.453959    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb" event={"ID":"e50f8853-6b18-4c60-94df-a1769d459c2b","Type":"ContainerDied","Data":"4e09bde10cfaff8bbbc329f0fda0371e6225ddf45523d4f147a53064cfd7a6bc"}
Sep 05 06:40:42.454007 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.453963    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"
Sep 05 06:40:42.462292 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.462268    2573 scope.go:117] "RemoveContainer" containerID="6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced"
Sep 05 06:40:42.462577 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:40:42.462558    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced\": container with ID starting with 6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced not found: ID does not exist" containerID="6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced"
Sep 05 06:40:42.462646 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.462583    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced"} err="failed to get container status \"6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced\": rpc error: code = NotFound desc = could not find container \"6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced\": container with ID starting with 6a640be504287fd2abd6c710988f61caf07c395c96d6848dd1e0a3c065c69ced not found: ID does not exist"
Sep 05 06:40:42.462646 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.462600    2573 scope.go:117] "RemoveContainer" containerID="8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb"
Sep 05 06:40:42.470817 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.470802    2573 scope.go:117] "RemoveContainer" containerID="8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb"
Sep 05 06:40:42.471102 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:40:42.471082    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb\": container with ID starting with 8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb not found: ID does not exist" containerID="8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb"
Sep 05 06:40:42.471159 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.471109    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb"} err="failed to get container status \"8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb\": rpc error: code = NotFound desc = could not find container \"8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb\": container with ID starting with 8bd98393e487e3ba2b5a8dc0a94a569df50df164073dc28a1c81f813b910c2fb not found: ID does not exist"
Sep 05 06:40:42.476561 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.476542    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"]
Sep 05 06:40:42.481149 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.481130    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8956b5-846947d97f-w28rc"]
Sep 05 06:40:42.493338 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.493312    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"]
Sep 05 06:40:42.497702 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.497683    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8956b5-7bcd464bdd-g6nbb"]
Sep 05 06:40:42.542587 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:42.542551    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:43.987739 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:43.987699    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl" podUID="bb94912d-19f3-4a54-8ec0-8c427a796550" containerName="main" containerID="cri-o://746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c" gracePeriod=45
Sep 05 06:40:44.156758 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.156730    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0593df1b-a65b-4445-ab71-4e71cc10bd00" path="/var/lib/kubelet/pods/0593df1b-a65b-4445-ab71-4e71cc10bd00/volumes"
Sep 05 06:40:44.157473 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.157451    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e50f8853-6b18-4c60-94df-a1769d459c2b" path="/var/lib/kubelet/pods/e50f8853-6b18-4c60-94df-a1769d459c2b/volumes"
Sep 05 06:40:44.259771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.259749    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:40:44.336431 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.336398    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-dshm\") pod \"bb94912d-19f3-4a54-8ec0-8c427a796550\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") "
Sep 05 06:40:44.336634 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.336481    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb94912d-19f3-4a54-8ec0-8c427a796550-tls-certs\") pod \"bb94912d-19f3-4a54-8ec0-8c427a796550\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") "
Sep 05 06:40:44.336634 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.336513    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6nx7\" (UniqueName: \"kubernetes.io/projected/bb94912d-19f3-4a54-8ec0-8c427a796550-kube-api-access-s6nx7\") pod \"bb94912d-19f3-4a54-8ec0-8c427a796550\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") "
Sep 05 06:40:44.336634 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.336570    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-tmp-dir\") pod \"bb94912d-19f3-4a54-8ec0-8c427a796550\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") "
Sep 05 06:40:44.336634 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.336609    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-model-cache\") pod \"bb94912d-19f3-4a54-8ec0-8c427a796550\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") "
Sep 05 06:40:44.336634 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.336631    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-home\") pod \"bb94912d-19f3-4a54-8ec0-8c427a796550\" (UID: \"bb94912d-19f3-4a54-8ec0-8c427a796550\") "
Sep 05 06:40:44.336931 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.336879    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-model-cache" (OuterVolumeSpecName: "model-cache") pod "bb94912d-19f3-4a54-8ec0-8c427a796550" (UID: "bb94912d-19f3-4a54-8ec0-8c427a796550"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:44.336996 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.336927    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bb94912d-19f3-4a54-8ec0-8c427a796550" (UID: "bb94912d-19f3-4a54-8ec0-8c427a796550"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:44.336996 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.336971    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-home" (OuterVolumeSpecName: "home") pod "bb94912d-19f3-4a54-8ec0-8c427a796550" (UID: "bb94912d-19f3-4a54-8ec0-8c427a796550"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:44.337094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.337004    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-tmp-dir\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:44.337094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.337024    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-model-cache\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:44.338838 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.338811    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-dshm" (OuterVolumeSpecName: "dshm") pod "bb94912d-19f3-4a54-8ec0-8c427a796550" (UID: "bb94912d-19f3-4a54-8ec0-8c427a796550"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:44.338838 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.338822    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb94912d-19f3-4a54-8ec0-8c427a796550-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bb94912d-19f3-4a54-8ec0-8c427a796550" (UID: "bb94912d-19f3-4a54-8ec0-8c427a796550"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:40:44.339020 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.338939    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb94912d-19f3-4a54-8ec0-8c427a796550-kube-api-access-s6nx7" (OuterVolumeSpecName: "kube-api-access-s6nx7") pod "bb94912d-19f3-4a54-8ec0-8c427a796550" (UID: "bb94912d-19f3-4a54-8ec0-8c427a796550"). InnerVolumeSpecName "kube-api-access-s6nx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:44.437869 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.437830    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-dshm\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:44.438035 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.437883    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb94912d-19f3-4a54-8ec0-8c427a796550-tls-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:44.438035 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.437895    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6nx7\" (UniqueName: \"kubernetes.io/projected/bb94912d-19f3-4a54-8ec0-8c427a796550-kube-api-access-s6nx7\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:44.438035 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.437904    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb94912d-19f3-4a54-8ec0-8c427a796550-home\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:44.470283 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.470253    2573 generic.go:356] "Generic (PLEG): container finished" podID="bb94912d-19f3-4a54-8ec0-8c427a796550" containerID="746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c" exitCode=0
Sep 05 06:40:44.470452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.470302    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"
Sep 05 06:40:44.470452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.470337    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl" event={"ID":"bb94912d-19f3-4a54-8ec0-8c427a796550","Type":"ContainerDied","Data":"746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c"}
Sep 05 06:40:44.470452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.470389    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl" event={"ID":"bb94912d-19f3-4a54-8ec0-8c427a796550","Type":"ContainerDied","Data":"c87ae68d1b123d6f05a4b715e026ab658a763905cd666089595109eefd256f6d"}
Sep 05 06:40:44.470452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.470407    2573 scope.go:117] "RemoveContainer" containerID="746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c"
Sep 05 06:40:44.479884 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.479865    2573 scope.go:117] "RemoveContainer" containerID="746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c"
Sep 05 06:40:44.480165 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:40:44.480146    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c\": container with ID starting with 746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c not found: ID does not exist" containerID="746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c"
Sep 05 06:40:44.480216 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.480172    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c"} err="failed to get container status \"746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c\": rpc error: code = NotFound desc = could not find container \"746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c\": container with ID starting with 746a24b457ee9ab2d24604b7fa2a5771e23d00e03d5baf73d5e1731ea9370c0c not found: ID does not exist"
Sep 05 06:40:44.494877 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.494796    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"]
Sep 05 06:40:44.497730 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:44.497704    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-c4e74764-kserve-6c6974b47-94mkl"]
Sep 05 06:40:45.712308 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:45.712275    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:46.156594 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:46.156559    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb94912d-19f3-4a54-8ec0-8c427a796550" path="/var/lib/kubelet/pods/bb94912d-19f3-4a54-8ec0-8c427a796550/volumes"
Sep 05 06:40:48.997110 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:48.997074    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:49.098616 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:49.098583    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:49.202057 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:49.202018    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:49.303302 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:49.303208    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:49.800305 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:49.800234    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:49.856467 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:49.856439    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"]
Sep 05 06:40:49.856689 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:49.856662    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx" podUID="f5e4ba35-1735-4b33-b2a6-ac355463b75e" containerName="maas-api" containerID="cri-o://975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266" gracePeriod=30
Sep 05 06:40:49.905445 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:49.905416    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:49.977461 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:49.977432    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"]
Sep 05 06:40:49.977660 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:49.977638    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78" podUID="10368558-1f6e-47b7-b096-512ef0fe4291" containerName="payload-pre-processing" containerID="cri-o://0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f" gracePeriod=30
Sep 05 06:40:50.019421 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.019386    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:50.116159 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.116098    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:50.119363 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.119338    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:50.194960 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.194932    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzxcz\" (UniqueName: \"kubernetes.io/projected/f5e4ba35-1735-4b33-b2a6-ac355463b75e-kube-api-access-wzxcz\") pod \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") "
Sep 05 06:40:50.195143 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.194973    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e4ba35-1735-4b33-b2a6-ac355463b75e-openshift-service-ca\") pod \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") "
Sep 05 06:40:50.195143 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.195024    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-maas-api-tls\") pod \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") "
Sep 05 06:40:50.195143 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.195057    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-metrics-certs\") pod \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\" (UID: \"f5e4ba35-1735-4b33-b2a6-ac355463b75e\") "
Sep 05 06:40:50.196005 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.195936    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5e4ba35-1735-4b33-b2a6-ac355463b75e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f5e4ba35-1735-4b33-b2a6-ac355463b75e" (UID: "f5e4ba35-1735-4b33-b2a6-ac355463b75e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:40:50.198733 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.198693    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5e4ba35-1735-4b33-b2a6-ac355463b75e-kube-api-access-wzxcz" (OuterVolumeSpecName: "kube-api-access-wzxcz") pod "f5e4ba35-1735-4b33-b2a6-ac355463b75e" (UID: "f5e4ba35-1735-4b33-b2a6-ac355463b75e"). InnerVolumeSpecName "kube-api-access-wzxcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:50.198836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.198788    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f5e4ba35-1735-4b33-b2a6-ac355463b75e" (UID: "f5e4ba35-1735-4b33-b2a6-ac355463b75e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:40:50.198836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.198810    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f5e4ba35-1735-4b33-b2a6-ac355463b75e" (UID: "f5e4ba35-1735-4b33-b2a6-ac355463b75e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:40:50.212706 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.212677    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:50.237320 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.237297    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:50.296374 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.296345    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27rrd\" (UniqueName: \"kubernetes.io/projected/10368558-1f6e-47b7-b096-512ef0fe4291-kube-api-access-27rrd\") pod \"10368558-1f6e-47b7-b096-512ef0fe4291\" (UID: \"10368558-1f6e-47b7-b096-512ef0fe4291\") "
Sep 05 06:40:50.296586 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.296437    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10368558-1f6e-47b7-b096-512ef0fe4291-plugins-config-volume\") pod \"10368558-1f6e-47b7-b096-512ef0fe4291\" (UID: \"10368558-1f6e-47b7-b096-512ef0fe4291\") "
Sep 05 06:40:50.296774 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.296755    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzxcz\" (UniqueName: \"kubernetes.io/projected/f5e4ba35-1735-4b33-b2a6-ac355463b75e-kube-api-access-wzxcz\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:50.296811 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.296779    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e4ba35-1735-4b33-b2a6-ac355463b75e-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:50.296811 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.296791    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:50.296811 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.296801    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5e4ba35-1735-4b33-b2a6-ac355463b75e-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:50.296942 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.296801    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10368558-1f6e-47b7-b096-512ef0fe4291-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "10368558-1f6e-47b7-b096-512ef0fe4291" (UID: "10368558-1f6e-47b7-b096-512ef0fe4291"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:40:50.298419 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.298402    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10368558-1f6e-47b7-b096-512ef0fe4291-kube-api-access-27rrd" (OuterVolumeSpecName: "kube-api-access-27rrd") pod "10368558-1f6e-47b7-b096-512ef0fe4291" (UID: "10368558-1f6e-47b7-b096-512ef0fe4291"). InnerVolumeSpecName "kube-api-access-27rrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:50.351362 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.351274    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:50.398157 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.398110    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27rrd\" (UniqueName: \"kubernetes.io/projected/10368558-1f6e-47b7-b096-512ef0fe4291-kube-api-access-27rrd\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:50.398157 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.398154    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10368558-1f6e-47b7-b096-512ef0fe4291-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:40:50.453679 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.453636    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:50.503947 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.503916    2573 generic.go:356] "Generic (PLEG): container finished" podID="f5e4ba35-1735-4b33-b2a6-ac355463b75e" containerID="975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266" exitCode=0
Sep 05 06:40:50.504126 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.503969    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"
Sep 05 06:40:50.504126 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.503997    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx" event={"ID":"f5e4ba35-1735-4b33-b2a6-ac355463b75e","Type":"ContainerDied","Data":"975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266"}
Sep 05 06:40:50.504126 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.504048    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx" event={"ID":"f5e4ba35-1735-4b33-b2a6-ac355463b75e","Type":"ContainerDied","Data":"ba53653daccf926eaa9060ad460c9ad9e962d76f9c070bf4571a7ddee3b63763"}
Sep 05 06:40:50.504126 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.504070    2573 scope.go:117] "RemoveContainer" containerID="975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266"
Sep 05 06:40:50.505424 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.505404    2573 generic.go:356] "Generic (PLEG): container finished" podID="10368558-1f6e-47b7-b096-512ef0fe4291" containerID="0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f" exitCode=0
Sep 05 06:40:50.505541 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.505453    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78" event={"ID":"10368558-1f6e-47b7-b096-512ef0fe4291","Type":"ContainerDied","Data":"0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f"}
Sep 05 06:40:50.505541 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.505483    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78" event={"ID":"10368558-1f6e-47b7-b096-512ef0fe4291","Type":"ContainerDied","Data":"266e8bd58d996dd2076ad5cee0fef13bc6c62bc39c22ff407ae82f8cc362ec3e"}
Sep 05 06:40:50.505541 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.505499    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"
Sep 05 06:40:50.513263 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.513244    2573 scope.go:117] "RemoveContainer" containerID="975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266"
Sep 05 06:40:50.513563 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:40:50.513539    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266\": container with ID starting with 975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266 not found: ID does not exist" containerID="975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266"
Sep 05 06:40:50.513652 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.513570    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266"} err="failed to get container status \"975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266\": rpc error: code = NotFound desc = could not find container \"975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266\": container with ID starting with 975e8035a4cba840d267f6f2f1bc92c451232890c0b821569c94a2a375533266 not found: ID does not exist"
Sep 05 06:40:50.513652 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.513587    2573 scope.go:117] "RemoveContainer" containerID="0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f"
Sep 05 06:40:50.522065 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.522050    2573 scope.go:117] "RemoveContainer" containerID="0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f"
Sep 05 06:40:50.522305 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:40:50.522286    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f\": container with ID starting with 0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f not found: ID does not exist" containerID="0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f"
Sep 05 06:40:50.522363 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.522313    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f"} err="failed to get container status \"0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f\": rpc error: code = NotFound desc = could not find container \"0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f\": container with ID starting with 0eb1c92eaf48abdd5d374593b278be2593fb2bc9d0c1f7208fb53d0b815c1a5f not found: ID does not exist"
Sep 05 06:40:50.535715 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.535696    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"]
Sep 05 06:40:50.547555 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.547535    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bba7714e-6848c698c9-gclzx"]
Sep 05 06:40:50.564239 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.564218    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"]
Sep 05 06:40:50.574575 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:50.574553    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bba7714e-7bc47c6f4b-vkz78"]
Sep 05 06:40:51.574824 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:51.574786    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:52.059188 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:52.059152    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:52.129394 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:52.129367    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:40:52.130026 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:52.130004    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:40:52.155627 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:52.155601    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10368558-1f6e-47b7-b096-512ef0fe4291" path="/var/lib/kubelet/pods/10368558-1f6e-47b7-b096-512ef0fe4291/volumes"
Sep 05 06:40:52.156017 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:52.156002    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5e4ba35-1735-4b33-b2a6-ac355463b75e" path="/var/lib/kubelet/pods/f5e4ba35-1735-4b33-b2a6-ac355463b75e/volumes"
Sep 05 06:40:52.414624 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:52.414549    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:52.738515 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:52.738425    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:53.618937 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:53.618904    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:53.705899 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:53.705845    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:54.748293 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.748259    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"]
Sep 05 06:40:54.748846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.748831    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e50f8853-6b18-4c60-94df-a1769d459c2b" containerName="maas-api"
Sep 05 06:40:54.748914 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.748866    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e50f8853-6b18-4c60-94df-a1769d459c2b" containerName="maas-api"
Sep 05 06:40:54.748914 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.748899    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb94912d-19f3-4a54-8ec0-8c427a796550" containerName="main"
Sep 05 06:40:54.748914 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.748908    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb94912d-19f3-4a54-8ec0-8c427a796550" containerName="main"
Sep 05 06:40:54.749026 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.748917    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0593df1b-a65b-4445-ab71-4e71cc10bd00" containerName="payload-processing"
Sep 05 06:40:54.749026 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.748927    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0593df1b-a65b-4445-ab71-4e71cc10bd00" containerName="payload-processing"
Sep 05 06:40:54.749026 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.748943    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e4ba35-1735-4b33-b2a6-ac355463b75e" containerName="maas-api"
Sep 05 06:40:54.749026 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.748951    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e4ba35-1735-4b33-b2a6-ac355463b75e" containerName="maas-api"
Sep 05 06:40:54.749026 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.748965    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10368558-1f6e-47b7-b096-512ef0fe4291" containerName="payload-pre-processing"
Sep 05 06:40:54.749026 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.748974    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="10368558-1f6e-47b7-b096-512ef0fe4291" containerName="payload-pre-processing"
Sep 05 06:40:54.749205 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.749055    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e50f8853-6b18-4c60-94df-a1769d459c2b" containerName="maas-api"
Sep 05 06:40:54.749205 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.749072    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0593df1b-a65b-4445-ab71-4e71cc10bd00" containerName="payload-processing"
Sep 05 06:40:54.749205 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.749083    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="10368558-1f6e-47b7-b096-512ef0fe4291" containerName="payload-pre-processing"
Sep 05 06:40:54.749205 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.749092    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb94912d-19f3-4a54-8ec0-8c427a796550" containerName="main"
Sep 05 06:40:54.749205 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.749100    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e4ba35-1735-4b33-b2a6-ac355463b75e" containerName="maas-api"
Sep 05 06:40:54.754250 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.754226    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:54.756785 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.756761    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-9f698c-serving-cert\""
Sep 05 06:40:54.765110 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.765088    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"]
Sep 05 06:40:54.801184 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.801145    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:54.840326 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.840293    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw8wg\" (UniqueName: \"kubernetes.io/projected/4e107042-4440-48ec-a728-15ad60318bc0-kube-api-access-vw8wg\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:54.840480 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.840344    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-maas-api-tls\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:54.840480 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.840414    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e107042-4440-48ec-a728-15ad60318bc0-openshift-service-ca\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:54.840480 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.840447    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-metrics-certs\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:54.902176 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.902136    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:54.941327 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.941290    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-maas-api-tls\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:54.941490 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.941338    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e107042-4440-48ec-a728-15ad60318bc0-openshift-service-ca\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:54.941490 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.941360    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-metrics-certs\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:54.941490 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:40:54.941450    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-serving-cert: secret "maas-api-e2e-explicit-9f698c-serving-cert" not found
Sep 05 06:40:54.941617 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.941455    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw8wg\" (UniqueName: \"kubernetes.io/projected/4e107042-4440-48ec-a728-15ad60318bc0-kube-api-access-vw8wg\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:54.941617 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:40:54.941524    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-maas-api-tls podName:4e107042-4440-48ec-a728-15ad60318bc0 nodeName:}" failed. No retries permitted until 2026-09-05 06:40:55.441503647 +0000 UTC m=+903.809550835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-maas-api-tls") pod "maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq" (UID: "4e107042-4440-48ec-a728-15ad60318bc0") : secret "maas-api-e2e-explicit-9f698c-serving-cert" not found
Sep 05 06:40:54.941983 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.941963    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e107042-4440-48ec-a728-15ad60318bc0-openshift-service-ca\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:54.943884 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.943845    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-metrics-certs\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:54.953745 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:54.953722    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw8wg\" (UniqueName: \"kubernetes.io/projected/4e107042-4440-48ec-a728-15ad60318bc0-kube-api-access-vw8wg\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:55.052749 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.052661    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:55.234992 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.234957    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:55.381001 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.380889    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:55.446163 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.446128    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-maas-api-tls\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:55.448757 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.448733    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-maas-api-tls\") pod \"maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:55.582316 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.582285    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:55.668348 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.668256    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:55.686284 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.686255    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:55.709651 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.709625    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"]
Sep 05 06:40:55.714250 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.714227    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:40:55.718101 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.717970    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-9f698c\""
Sep 05 06:40:55.725336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.725297    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"]
Sep 05 06:40:55.730877 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.730743    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"]
Sep 05 06:40:55.730877 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.730875    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:40:55.733704 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.733686    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-9f698c-dockercfg-4s8b9\""
Sep 05 06:40:55.745080 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.745050    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"]
Sep 05 06:40:55.850683 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.850647    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzl6b\" (UniqueName: \"kubernetes.io/projected/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-kube-api-access-gzl6b\") pod \"payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6\" (UID: \"c4d8053a-ccbb-4802-81a6-49a0678fb7b9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:40:55.851138 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.850686    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6\" (UID: \"c4d8053a-ccbb-4802-81a6-49a0678fb7b9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:40:55.851138 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.850774    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfxg5\" (UniqueName: \"kubernetes.io/projected/b3ea980e-1782-429d-8714-25ff791a8a8d-kube-api-access-dfxg5\") pod \"payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss\" (UID: \"b3ea980e-1782-429d-8714-25ff791a8a8d\") " pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:40:55.851138 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.850903    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3ea980e-1782-429d-8714-25ff791a8a8d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss\" (UID: \"b3ea980e-1782-429d-8714-25ff791a8a8d\") " pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:40:55.944622 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.944589    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"]
Sep 05 06:40:55.945606 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:40:55.945578    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e107042_4440_48ec_a728_15ad60318bc0.slice/crio-9daf5435450f89253853164de234304f856f056e39cced0ed13f677eff3cb23d WatchSource:0}: Error finding container 9daf5435450f89253853164de234304f856f056e39cced0ed13f677eff3cb23d: Status 404 returned error can't find the container with id 9daf5435450f89253853164de234304f856f056e39cced0ed13f677eff3cb23d
Sep 05 06:40:55.951870 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.951819    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfxg5\" (UniqueName: \"kubernetes.io/projected/b3ea980e-1782-429d-8714-25ff791a8a8d-kube-api-access-dfxg5\") pod \"payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss\" (UID: \"b3ea980e-1782-429d-8714-25ff791a8a8d\") " pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:40:55.951962 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.951918    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3ea980e-1782-429d-8714-25ff791a8a8d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss\" (UID: \"b3ea980e-1782-429d-8714-25ff791a8a8d\") " pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:40:55.952058 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.952036    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzl6b\" (UniqueName: \"kubernetes.io/projected/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-kube-api-access-gzl6b\") pod \"payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6\" (UID: \"c4d8053a-ccbb-4802-81a6-49a0678fb7b9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:40:55.952171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.952074    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6\" (UID: \"c4d8053a-ccbb-4802-81a6-49a0678fb7b9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:40:55.952817 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.952767    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3ea980e-1782-429d-8714-25ff791a8a8d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss\" (UID: \"b3ea980e-1782-429d-8714-25ff791a8a8d\") " pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:40:55.952817 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.952808    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6\" (UID: \"c4d8053a-ccbb-4802-81a6-49a0678fb7b9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:40:55.960987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.960966    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfxg5\" (UniqueName: \"kubernetes.io/projected/b3ea980e-1782-429d-8714-25ff791a8a8d-kube-api-access-dfxg5\") pod \"payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss\" (UID: \"b3ea980e-1782-429d-8714-25ff791a8a8d\") " pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:40:55.962456 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:55.962436    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzl6b\" (UniqueName: \"kubernetes.io/projected/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-kube-api-access-gzl6b\") pod \"payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6\" (UID: \"c4d8053a-ccbb-4802-81a6-49a0678fb7b9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:40:56.032574 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.032544    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:40:56.046917 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.046890    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:40:56.310077 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.310038    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"]
Sep 05 06:40:56.311656 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:40:56.311632    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d8053a_ccbb_4802_81a6_49a0678fb7b9.slice/crio-491cb3252c0a8651e43398c4fb4f5267011351c70dbc7d505459d9748b24f7f5 WatchSource:0}: Error finding container 491cb3252c0a8651e43398c4fb4f5267011351c70dbc7d505459d9748b24f7f5: Status 404 returned error can't find the container with id 491cb3252c0a8651e43398c4fb4f5267011351c70dbc7d505459d9748b24f7f5
Sep 05 06:40:56.329239 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.329206    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"]
Sep 05 06:40:56.330294 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:40:56.330274    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ea980e_1782_429d_8714_25ff791a8a8d.slice/crio-98ab2b19017fc9e6de6d058d49d28afae5810b1712b52e4c9af3e8ca9e27355d WatchSource:0}: Error finding container 98ab2b19017fc9e6de6d058d49d28afae5810b1712b52e4c9af3e8ca9e27355d: Status 404 returned error can't find the container with id 98ab2b19017fc9e6de6d058d49d28afae5810b1712b52e4c9af3e8ca9e27355d
Sep 05 06:40:56.537831 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.537802    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6" event={"ID":"c4d8053a-ccbb-4802-81a6-49a0678fb7b9","Type":"ContainerStarted","Data":"84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944"}
Sep 05 06:40:56.537952 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.537877    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6" event={"ID":"c4d8053a-ccbb-4802-81a6-49a0678fb7b9","Type":"ContainerStarted","Data":"491cb3252c0a8651e43398c4fb4f5267011351c70dbc7d505459d9748b24f7f5"}
Sep 05 06:40:56.537952 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.537906    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:40:56.539402 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.539379    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss" event={"ID":"b3ea980e-1782-429d-8714-25ff791a8a8d","Type":"ContainerStarted","Data":"fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7"}
Sep 05 06:40:56.539494 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.539424    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss" event={"ID":"b3ea980e-1782-429d-8714-25ff791a8a8d","Type":"ContainerStarted","Data":"98ab2b19017fc9e6de6d058d49d28afae5810b1712b52e4c9af3e8ca9e27355d"}
Sep 05 06:40:56.539643 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.539619    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:40:56.541493 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.541468    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq" event={"ID":"4e107042-4440-48ec-a728-15ad60318bc0","Type":"ContainerStarted","Data":"9daf5435450f89253853164de234304f856f056e39cced0ed13f677eff3cb23d"}
Sep 05 06:40:56.541606 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.541589    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:40:56.560106 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.560010    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6" podStartSLOduration=1.559991138 podStartE2EDuration="1.559991138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:56.557169254 +0000 UTC m=+904.925216466" watchObservedRunningTime="2026-09-05 06:40:56.559991138 +0000 UTC m=+904.928038349"
Sep 05 06:40:56.575741 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.575688    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss" podStartSLOduration=1.575675884 podStartE2EDuration="1.575675884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:56.575005174 +0000 UTC m=+904.943052384" watchObservedRunningTime="2026-09-05 06:40:56.575675884 +0000 UTC m=+904.943723096"
Sep 05 06:40:56.714637 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.714571    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq" podStartSLOduration=2.184860012 podStartE2EDuration="2.714547172s" totalImagesPullingTime="529.68716ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:56.592910357 +0000 UTC m=+904.960957569" watchObservedRunningTime="2026-09-05 06:40:56.714547172 +0000 UTC m=+905.082594383"
Sep 05 06:40:56.717053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.717028    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"]
Sep 05 06:40:56.722251 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.722220    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.725214 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.724828    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-13503c3c-serving-cert\""
Sep 05 06:40:56.728242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.728162    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"]
Sep 05 06:40:56.751602 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.751575    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:56.820260 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.820115    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:56.862371 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.862337    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-maas-api-tls\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.862797 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.862380    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-metrics-certs\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.862797 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.862436    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/266c70df-ed81-49f6-8b42-77053b0a6d31-openshift-service-ca\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.862797 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.862527    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f9sz\" (UniqueName: \"kubernetes.io/projected/266c70df-ed81-49f6-8b42-77053b0a6d31-kube-api-access-6f9sz\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.958497 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.958455    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:56.963567 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.963543    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f9sz\" (UniqueName: \"kubernetes.io/projected/266c70df-ed81-49f6-8b42-77053b0a6d31-kube-api-access-6f9sz\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.963725 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.963635    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-maas-api-tls\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.963725 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.963662    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-metrics-certs\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.963725 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.963719    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/266c70df-ed81-49f6-8b42-77053b0a6d31-openshift-service-ca\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.964540 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.964513    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/266c70df-ed81-49f6-8b42-77053b0a6d31-openshift-service-ca\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.966261 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.966237    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-maas-api-tls\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.966370 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.966323    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-metrics-certs\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:56.972655 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:56.972631    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f9sz\" (UniqueName: \"kubernetes.io/projected/266c70df-ed81-49f6-8b42-77053b0a6d31-kube-api-access-6f9sz\") pod \"maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:57.038796 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:57.038767    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:40:57.290724 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:57.290681    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:57.313238 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:57.313206    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"]
Sep 05 06:40:57.316907 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:40:57.316879    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266c70df_ed81_49f6_8b42_77053b0a6d31.slice/crio-17722a64a34c6965ddb168723bff9128486adc0f67211fde581c9c9b72ac0ce3 WatchSource:0}: Error finding container 17722a64a34c6965ddb168723bff9128486adc0f67211fde581c9c9b72ac0ce3: Status 404 returned error can't find the container with id 17722a64a34c6965ddb168723bff9128486adc0f67211fde581c9c9b72ac0ce3
Sep 05 06:40:57.392056 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:57.391978    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:57.548349 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:57.548312    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m" event={"ID":"266c70df-ed81-49f6-8b42-77053b0a6d31","Type":"ContainerStarted","Data":"17722a64a34c6965ddb168723bff9128486adc0f67211fde581c9c9b72ac0ce3"}
Sep 05 06:40:57.550011 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:57.549980    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq" event={"ID":"4e107042-4440-48ec-a728-15ad60318bc0","Type":"ContainerStarted","Data":"c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143"}
Sep 05 06:40:57.766939 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:57.766900    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:57.893377 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:57.893293    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:58.556732 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:58.556692    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m" event={"ID":"266c70df-ed81-49f6-8b42-77053b0a6d31","Type":"ContainerStarted","Data":"7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e"}
Sep 05 06:40:58.578950 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:58.578901    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m" podStartSLOduration=2.290917692 podStartE2EDuration="2.578885273s" totalImagesPullingTime="287.967581ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:58.577900291 +0000 UTC m=+906.945947500" watchObservedRunningTime="2026-09-05 06:40:58.578885273 +0000 UTC m=+906.946932483"
Sep 05 06:40:58.657150 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:58.657121    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:40:59.562107 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:40:59.562075    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:41:03.561961 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:03.561933    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:41:05.574592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:05.574565    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:41:07.552723 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:07.552680    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:41:07.553184 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:07.552809    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:41:16.059906 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:16.059868    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:18.209766 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:18.209726    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:18.602429 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:18.602386    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:26.949451 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:26.949388    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"]
Sep 05 06:41:26.949954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:26.949629    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m" podUID="266c70df-ed81-49f6-8b42-77053b0a6d31" containerName="maas-api" containerID="cri-o://7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e" gracePeriod=30
Sep 05 06:41:26.971197 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:26.971169    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:27.056502 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.056458    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:27.125378 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.125341    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:27.199590 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.199538    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:41:27.233843 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.233811    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:27.258075 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.258052    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-maas-api-tls\") pod \"266c70df-ed81-49f6-8b42-77053b0a6d31\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") "
Sep 05 06:41:27.258196 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.258141    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-metrics-certs\") pod \"266c70df-ed81-49f6-8b42-77053b0a6d31\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") "
Sep 05 06:41:27.258196 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.258178    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/266c70df-ed81-49f6-8b42-77053b0a6d31-openshift-service-ca\") pod \"266c70df-ed81-49f6-8b42-77053b0a6d31\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") "
Sep 05 06:41:27.258310 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.258260    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f9sz\" (UniqueName: \"kubernetes.io/projected/266c70df-ed81-49f6-8b42-77053b0a6d31-kube-api-access-6f9sz\") pod \"266c70df-ed81-49f6-8b42-77053b0a6d31\" (UID: \"266c70df-ed81-49f6-8b42-77053b0a6d31\") "
Sep 05 06:41:27.258613 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.258589    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/266c70df-ed81-49f6-8b42-77053b0a6d31-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "266c70df-ed81-49f6-8b42-77053b0a6d31" (UID: "266c70df-ed81-49f6-8b42-77053b0a6d31"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:41:27.260242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.260213    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "266c70df-ed81-49f6-8b42-77053b0a6d31" (UID: "266c70df-ed81-49f6-8b42-77053b0a6d31"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:41:27.260336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.260252    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "266c70df-ed81-49f6-8b42-77053b0a6d31" (UID: "266c70df-ed81-49f6-8b42-77053b0a6d31"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:41:27.260336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.260311    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/266c70df-ed81-49f6-8b42-77053b0a6d31-kube-api-access-6f9sz" (OuterVolumeSpecName: "kube-api-access-6f9sz") pod "266c70df-ed81-49f6-8b42-77053b0a6d31" (UID: "266c70df-ed81-49f6-8b42-77053b0a6d31"). InnerVolumeSpecName "kube-api-access-6f9sz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:41:27.297697 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.297673    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:27.360201 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.360174    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:41:27.360201 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.360203    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/266c70df-ed81-49f6-8b42-77053b0a6d31-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:41:27.360427 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.360215    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f9sz\" (UniqueName: \"kubernetes.io/projected/266c70df-ed81-49f6-8b42-77053b0a6d31-kube-api-access-6f9sz\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:41:27.360427 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.360224    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/266c70df-ed81-49f6-8b42-77053b0a6d31-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:41:27.388417 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.388383    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:27.702934 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.702901    2573 generic.go:356] "Generic (PLEG): container finished" podID="266c70df-ed81-49f6-8b42-77053b0a6d31" containerID="7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e" exitCode=0
Sep 05 06:41:27.703121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.702946    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"
Sep 05 06:41:27.703121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.702989    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m" event={"ID":"266c70df-ed81-49f6-8b42-77053b0a6d31","Type":"ContainerDied","Data":"7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e"}
Sep 05 06:41:27.703121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.703051    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m" event={"ID":"266c70df-ed81-49f6-8b42-77053b0a6d31","Type":"ContainerDied","Data":"17722a64a34c6965ddb168723bff9128486adc0f67211fde581c9c9b72ac0ce3"}
Sep 05 06:41:27.703121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.703102    2573 scope.go:117] "RemoveContainer" containerID="7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e"
Sep 05 06:41:27.712665 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.712651    2573 scope.go:117] "RemoveContainer" containerID="7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e"
Sep 05 06:41:27.712945 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:41:27.712927    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e\": container with ID starting with 7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e not found: ID does not exist" containerID="7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e"
Sep 05 06:41:27.712997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.712953    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e"} err="failed to get container status \"7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e\": rpc error: code = NotFound desc = could not find container \"7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e\": container with ID starting with 7dec01ff7624193b63799c8fcda8a61c173868ee2833025e0b12ff875950622e not found: ID does not exist"
Sep 05 06:41:27.728436 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.728413    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"]
Sep 05 06:41:27.742173 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:27.742153    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-13503c3c-65f6fd6f74-4lv7m"]
Sep 05 06:41:28.063175 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:28.063141    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:28.156914 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:28.156885    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="266c70df-ed81-49f6-8b42-77053b0a6d31" path="/var/lib/kubelet/pods/266c70df-ed81-49f6-8b42-77053b0a6d31/volumes"
Sep 05 06:41:28.158834 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:28.158810    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:28.262291 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:28.262263    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:34.787393 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:34.787357    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:39.076820 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:39.076776    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:41.307101 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:41.307059    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:41.413384 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:41.413349    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:41.514667 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:41.514626    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:41.617909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:41.617829    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:42.028919 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:42.028884    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:53.687622 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.687581    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"]
Sep 05 06:41:53.688105 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.688090    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="266c70df-ed81-49f6-8b42-77053b0a6d31" containerName="maas-api"
Sep 05 06:41:53.688160 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.688108    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="266c70df-ed81-49f6-8b42-77053b0a6d31" containerName="maas-api"
Sep 05 06:41:53.688197 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.688191    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="266c70df-ed81-49f6-8b42-77053b0a6d31" containerName="maas-api"
Sep 05 06:41:53.692958 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.692938    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.696962 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.696930    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-9f698c\"/\"default-dockercfg-mpphw\""
Sep 05 06:41:53.697083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.696959    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-9f698c\"/\"kube-root-ca.crt\""
Sep 05 06:41:53.697083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.696978    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-9f698c\"/\"explicit-model-9f698c-kserve-self-signed-certs\""
Sep 05 06:41:53.697083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.696961    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-9f698c\"/\"openshift-service-ca.crt\""
Sep 05 06:41:53.701917 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.701896    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"]
Sep 05 06:41:53.710292 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.710271    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-dshm\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.710394 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.710297    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-model-cache\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.710394 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.710316    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmhvq\" (UniqueName: \"kubernetes.io/projected/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-kube-api-access-dmhvq\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.710467 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.710394    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tmp-dir\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.710467 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.710411    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tls-certs\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.710535 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.710478    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-home\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.713418 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.713393    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:53.811179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.811142    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-home\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.811371 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.811221    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-dshm\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.811371 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.811248    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-model-cache\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.811371 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.811272    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmhvq\" (UniqueName: \"kubernetes.io/projected/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-kube-api-access-dmhvq\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.811371 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.811365    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tmp-dir\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.811599 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.811388    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tls-certs\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.811599 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.811519    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-home\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.811707 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.811662    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-model-cache\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.811707 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.811698    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tmp-dir\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.813405 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.813387    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-dshm\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.813780 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.813760    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tls-certs\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:53.821037 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:53.821016    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmhvq\" (UniqueName: \"kubernetes.io/projected/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-kube-api-access-dmhvq\") pod \"explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") " pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:54.003370 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:54.003330    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:41:54.278905 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:54.278822    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"]
Sep 05 06:41:54.281513 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:41:54.281485    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca27ba8c_b11d_4496_99e6_bee61f6b8c8e.slice/crio-1a9465f946002c6f4e291b3813ac101285d7914335332205089aeb6851ae6ac6 WatchSource:0}: Error finding container 1a9465f946002c6f4e291b3813ac101285d7914335332205089aeb6851ae6ac6: Status 404 returned error can't find the container with id 1a9465f946002c6f4e291b3813ac101285d7914335332205089aeb6851ae6ac6
Sep 05 06:41:54.835567 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:54.835533    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h" event={"ID":"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e","Type":"ContainerStarted","Data":"198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0"}
Sep 05 06:41:54.835567 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:54.835578    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h" event={"ID":"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e","Type":"ContainerStarted","Data":"1a9465f946002c6f4e291b3813ac101285d7914335332205089aeb6851ae6ac6"}
Sep 05 06:41:54.854968 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:54.854916    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h" podStartSLOduration=1.854896509 podStartE2EDuration="1.854896509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:41:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:41:54.854241543 +0000 UTC m=+963.222288776" watchObservedRunningTime="2026-09-05 06:41:54.854896509 +0000 UTC m=+963.222943718"
Sep 05 06:41:57.286370 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:57.286336    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"]
Sep 05 06:41:57.308685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:57.308654    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:41:57.535960 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:41:57.535922    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:04.003992 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:04.003945    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:42:07.908712 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:07.908683    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"]
Sep 05 06:42:07.909202 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:07.908912    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq" podUID="4e107042-4440-48ec-a728-15ad60318bc0" containerName="maas-api" containerID="cri-o://c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143" gracePeriod=30
Sep 05 06:42:07.942543 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:07.942508    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:07.975467 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:07.975426    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"]
Sep 05 06:42:07.975720 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:07.975689    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss" podUID="b3ea980e-1782-429d-8714-25ff791a8a8d" containerName="payload-processing" containerID="cri-o://fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7" gracePeriod=30
Sep 05 06:42:07.983424 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:07.983399    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"]
Sep 05 06:42:07.983657 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:07.983617    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6" podUID="c4d8053a-ccbb-4802-81a6-49a0678fb7b9" containerName="payload-pre-processing" containerID="cri-o://84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944" gracePeriod=30
Sep 05 06:42:08.021870 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.021828    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:08.092046 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.091953    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:08.168792 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.162786    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:08.230357 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.230328    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:08.251658 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.251630    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:42:08.268256 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.268227    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:42:08.280480 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.280455    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:42:08.344583 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.344549    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e107042-4440-48ec-a728-15ad60318bc0-openshift-service-ca\") pod \"4e107042-4440-48ec-a728-15ad60318bc0\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") "
Sep 05 06:42:08.344762 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.344604    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-metrics-certs\") pod \"4e107042-4440-48ec-a728-15ad60318bc0\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") "
Sep 05 06:42:08.344762 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.344634    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-maas-api-tls\") pod \"4e107042-4440-48ec-a728-15ad60318bc0\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") "
Sep 05 06:42:08.344903 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.344768    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3ea980e-1782-429d-8714-25ff791a8a8d-plugins-config-volume\") pod \"b3ea980e-1782-429d-8714-25ff791a8a8d\" (UID: \"b3ea980e-1782-429d-8714-25ff791a8a8d\") "
Sep 05 06:42:08.344903 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.344887    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-plugins-config-volume\") pod \"c4d8053a-ccbb-4802-81a6-49a0678fb7b9\" (UID: \"c4d8053a-ccbb-4802-81a6-49a0678fb7b9\") "
Sep 05 06:42:08.345037 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.344940    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vw8wg\" (UniqueName: \"kubernetes.io/projected/4e107042-4440-48ec-a728-15ad60318bc0-kube-api-access-vw8wg\") pod \"4e107042-4440-48ec-a728-15ad60318bc0\" (UID: \"4e107042-4440-48ec-a728-15ad60318bc0\") "
Sep 05 06:42:08.345037 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.344987    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzl6b\" (UniqueName: \"kubernetes.io/projected/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-kube-api-access-gzl6b\") pod \"c4d8053a-ccbb-4802-81a6-49a0678fb7b9\" (UID: \"c4d8053a-ccbb-4802-81a6-49a0678fb7b9\") "
Sep 05 06:42:08.345037 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.345015    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfxg5\" (UniqueName: \"kubernetes.io/projected/b3ea980e-1782-429d-8714-25ff791a8a8d-kube-api-access-dfxg5\") pod \"b3ea980e-1782-429d-8714-25ff791a8a8d\" (UID: \"b3ea980e-1782-429d-8714-25ff791a8a8d\") "
Sep 05 06:42:08.345294 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.344981    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e107042-4440-48ec-a728-15ad60318bc0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4e107042-4440-48ec-a728-15ad60318bc0" (UID: "4e107042-4440-48ec-a728-15ad60318bc0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:42:08.345377 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.345310    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c4d8053a-ccbb-4802-81a6-49a0678fb7b9" (UID: "c4d8053a-ccbb-4802-81a6-49a0678fb7b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:42:08.345377 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.345257    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3ea980e-1782-429d-8714-25ff791a8a8d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b3ea980e-1782-429d-8714-25ff791a8a8d" (UID: "b3ea980e-1782-429d-8714-25ff791a8a8d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:42:08.345513 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.345485    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:08.345513 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.345514    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e107042-4440-48ec-a728-15ad60318bc0-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:08.345696 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.345530    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3ea980e-1782-429d-8714-25ff791a8a8d-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:08.347040 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.347015    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4e107042-4440-48ec-a728-15ad60318bc0" (UID: "4e107042-4440-48ec-a728-15ad60318bc0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:42:08.347134 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.347036    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4e107042-4440-48ec-a728-15ad60318bc0" (UID: "4e107042-4440-48ec-a728-15ad60318bc0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:42:08.347134 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.347045    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e107042-4440-48ec-a728-15ad60318bc0-kube-api-access-vw8wg" (OuterVolumeSpecName: "kube-api-access-vw8wg") pod "4e107042-4440-48ec-a728-15ad60318bc0" (UID: "4e107042-4440-48ec-a728-15ad60318bc0"). InnerVolumeSpecName "kube-api-access-vw8wg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:42:08.347492 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.347466    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-kube-api-access-gzl6b" (OuterVolumeSpecName: "kube-api-access-gzl6b") pod "c4d8053a-ccbb-4802-81a6-49a0678fb7b9" (UID: "c4d8053a-ccbb-4802-81a6-49a0678fb7b9"). InnerVolumeSpecName "kube-api-access-gzl6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:42:08.347580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.347569    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3ea980e-1782-429d-8714-25ff791a8a8d-kube-api-access-dfxg5" (OuterVolumeSpecName: "kube-api-access-dfxg5") pod "b3ea980e-1782-429d-8714-25ff791a8a8d" (UID: "b3ea980e-1782-429d-8714-25ff791a8a8d"). InnerVolumeSpecName "kube-api-access-dfxg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:42:08.446721 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.446639    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw8wg\" (UniqueName: \"kubernetes.io/projected/4e107042-4440-48ec-a728-15ad60318bc0-kube-api-access-vw8wg\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:08.446721 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.446668    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzl6b\" (UniqueName: \"kubernetes.io/projected/c4d8053a-ccbb-4802-81a6-49a0678fb7b9-kube-api-access-gzl6b\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:08.446721 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.446677    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfxg5\" (UniqueName: \"kubernetes.io/projected/b3ea980e-1782-429d-8714-25ff791a8a8d-kube-api-access-dfxg5\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:08.446721 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.446687    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:08.446721 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.446696    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e107042-4440-48ec-a728-15ad60318bc0-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:08.520304 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.520270    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:08.908315 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.908276    2573 generic.go:356] "Generic (PLEG): container finished" podID="c4d8053a-ccbb-4802-81a6-49a0678fb7b9" containerID="84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944" exitCode=0
Sep 05 06:42:08.908531 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.908368    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"
Sep 05 06:42:08.908531 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.908402    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6" event={"ID":"c4d8053a-ccbb-4802-81a6-49a0678fb7b9","Type":"ContainerDied","Data":"84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944"}
Sep 05 06:42:08.908531 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.908486    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6" event={"ID":"c4d8053a-ccbb-4802-81a6-49a0678fb7b9","Type":"ContainerDied","Data":"491cb3252c0a8651e43398c4fb4f5267011351c70dbc7d505459d9748b24f7f5"}
Sep 05 06:42:08.908531 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.908516    2573 scope.go:117] "RemoveContainer" containerID="84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944"
Sep 05 06:42:08.909986 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.909916    2573 generic.go:356] "Generic (PLEG): container finished" podID="b3ea980e-1782-429d-8714-25ff791a8a8d" containerID="fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7" exitCode=0
Sep 05 06:42:08.909986 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.909953    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"
Sep 05 06:42:08.910329 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.909997    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss" event={"ID":"b3ea980e-1782-429d-8714-25ff791a8a8d","Type":"ContainerDied","Data":"fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7"}
Sep 05 06:42:08.910329 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.910053    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss" event={"ID":"b3ea980e-1782-429d-8714-25ff791a8a8d","Type":"ContainerDied","Data":"98ab2b19017fc9e6de6d058d49d28afae5810b1712b52e4c9af3e8ca9e27355d"}
Sep 05 06:42:08.911552 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.911528    2573 generic.go:356] "Generic (PLEG): container finished" podID="4e107042-4440-48ec-a728-15ad60318bc0" containerID="c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143" exitCode=0
Sep 05 06:42:08.911668 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.911563    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"
Sep 05 06:42:08.911668 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.911606    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq" event={"ID":"4e107042-4440-48ec-a728-15ad60318bc0","Type":"ContainerDied","Data":"c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143"}
Sep 05 06:42:08.911668 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.911653    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq" event={"ID":"4e107042-4440-48ec-a728-15ad60318bc0","Type":"ContainerDied","Data":"9daf5435450f89253853164de234304f856f056e39cced0ed13f677eff3cb23d"}
Sep 05 06:42:08.919259 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.919236    2573 scope.go:117] "RemoveContainer" containerID="84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944"
Sep 05 06:42:08.919573 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:42:08.919554    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944\": container with ID starting with 84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944 not found: ID does not exist" containerID="84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944"
Sep 05 06:42:08.919641 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.919582    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944"} err="failed to get container status \"84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944\": rpc error: code = NotFound desc = could not find container \"84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944\": container with ID starting with 84d6b292f3ba7204c0039b81cabd717ee1de23cf83d76f072578f132b825e944 not found: ID does not exist"
Sep 05 06:42:08.919641 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.919599    2573 scope.go:117] "RemoveContainer" containerID="fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7"
Sep 05 06:42:08.939159 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.939141    2573 scope.go:117] "RemoveContainer" containerID="fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7"
Sep 05 06:42:08.939488 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:42:08.939467    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7\": container with ID starting with fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7 not found: ID does not exist" containerID="fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7"
Sep 05 06:42:08.939566 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.939496    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7"} err="failed to get container status \"fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7\": rpc error: code = NotFound desc = could not find container \"fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7\": container with ID starting with fb822857e5908d2d18eb9cf8bb4c65871c9d803e0d9d64473f26fa1aa0f96ff7 not found: ID does not exist"
Sep 05 06:42:08.939566 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.939515    2573 scope.go:117] "RemoveContainer" containerID="c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143"
Sep 05 06:42:08.948597 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.948577    2573 scope.go:117] "RemoveContainer" containerID="c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143"
Sep 05 06:42:08.948913 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:42:08.948889    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143\": container with ID starting with c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143 not found: ID does not exist" containerID="c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143"
Sep 05 06:42:08.948990 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.948921    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143"} err="failed to get container status \"c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143\": rpc error: code = NotFound desc = could not find container \"c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143\": container with ID starting with c4661be6140feb30b64034ef2565db166777110987c8a992fcce95c901229143 not found: ID does not exist"
Sep 05 06:42:08.949424 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.949405    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"]
Sep 05 06:42:08.955974 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.955944    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-9f698c-6df47f5c66-slfg6"]
Sep 05 06:42:08.970593 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.970560    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"]
Sep 05 06:42:08.976106 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.976080    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-9f698c-7d54f5db49-wndss"]
Sep 05 06:42:08.987938 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.987912    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"]
Sep 05 06:42:08.996132 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:08.996102    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9f698c-744f65b8f5-s7jjq"]
Sep 05 06:42:10.156421 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:10.156377    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e107042-4440-48ec-a728-15ad60318bc0" path="/var/lib/kubelet/pods/4e107042-4440-48ec-a728-15ad60318bc0/volumes"
Sep 05 06:42:10.157075 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:10.157055    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3ea980e-1782-429d-8714-25ff791a8a8d" path="/var/lib/kubelet/pods/b3ea980e-1782-429d-8714-25ff791a8a8d/volumes"
Sep 05 06:42:10.157414 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:10.157401    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4d8053a-ccbb-4802-81a6-49a0678fb7b9" path="/var/lib/kubelet/pods/c4d8053a-ccbb-4802-81a6-49a0678fb7b9/volumes"
Sep 05 06:42:12.306868 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.306805    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h" podUID="ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" containerName="main" containerID="cri-o://198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0" gracePeriod=45
Sep 05 06:42:12.564194 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.564136    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:42:12.686843 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.686809    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-dshm\") pod \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") "
Sep 05 06:42:12.687036 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.686904    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-home\") pod \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") "
Sep 05 06:42:12.687036 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.686941    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-model-cache\") pod \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") "
Sep 05 06:42:12.687036 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.686964    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmhvq\" (UniqueName: \"kubernetes.io/projected/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-kube-api-access-dmhvq\") pod \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") "
Sep 05 06:42:12.687036 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.687004    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tls-certs\") pod \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") "
Sep 05 06:42:12.687036 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.687035    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tmp-dir\") pod \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\" (UID: \"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e\") "
Sep 05 06:42:12.687305 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.687183    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-model-cache" (OuterVolumeSpecName: "model-cache") pod "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" (UID: "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:12.687305 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.687195    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-home" (OuterVolumeSpecName: "home") pod "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" (UID: "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:12.687414 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.687340    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" (UID: "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:12.687468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.687455    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-home\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:12.687526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.687468    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-model-cache\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:12.687526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.687478    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tmp-dir\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:12.689092 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.689066    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-dshm" (OuterVolumeSpecName: "dshm") pod "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" (UID: "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:12.689092 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.689082    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-kube-api-access-dmhvq" (OuterVolumeSpecName: "kube-api-access-dmhvq") pod "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" (UID: "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e"). InnerVolumeSpecName "kube-api-access-dmhvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:42:12.689252 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.689104    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" (UID: "ca27ba8c-b11d-4496-99e6-bee61f6b8c8e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:42:12.788458 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.788408    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-tls-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:12.788458 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.788452    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-dshm\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:12.788458 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.788462    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmhvq\" (UniqueName: \"kubernetes.io/projected/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e-kube-api-access-dmhvq\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:42:12.938735 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.938634    2573 generic.go:356] "Generic (PLEG): container finished" podID="ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" containerID="198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0" exitCode=0
Sep 05 06:42:12.938735 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.938690    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"
Sep 05 06:42:12.938938 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.938729    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h" event={"ID":"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e","Type":"ContainerDied","Data":"198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0"}
Sep 05 06:42:12.938938 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.938775    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h" event={"ID":"ca27ba8c-b11d-4496-99e6-bee61f6b8c8e","Type":"ContainerDied","Data":"1a9465f946002c6f4e291b3813ac101285d7914335332205089aeb6851ae6ac6"}
Sep 05 06:42:12.938938 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.938792    2573 scope.go:117] "RemoveContainer" containerID="198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0"
Sep 05 06:42:12.947653 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.947637    2573 scope.go:117] "RemoveContainer" containerID="198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0"
Sep 05 06:42:12.947939 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:42:12.947919    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0\": container with ID starting with 198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0 not found: ID does not exist" containerID="198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0"
Sep 05 06:42:12.947992 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.947946    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0"} err="failed to get container status \"198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0\": rpc error: code = NotFound desc = could not find container \"198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0\": container with ID starting with 198fcb69321446ea6b80da06662fb29b3bddbfa15fc2d2d976f497267e1f03d0 not found: ID does not exist"
Sep 05 06:42:12.960225 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.960197    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"]
Sep 05 06:42:12.964787 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:12.964764    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-9f698c/explicit-model-9f698c-kserve-7b6b5bc6b7-8b48h"]
Sep 05 06:42:14.155980 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:14.155908    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" path="/var/lib/kubelet/pods/ca27ba8c-b11d-4496-99e6-bee61f6b8c8e/volumes"
Sep 05 06:42:14.636173 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:14.636135    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:14.736244 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:14.736188    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:14.853177 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:14.853144    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:15.273198 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:15.273168    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:15.372761 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:15.372725    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:15.534376 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:15.534286    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:15.999743 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:15.999716    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:16.319740 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:16.319657    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:16.444918 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:16.444885    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:17.413240 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:17.413198    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:17.511800 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:17.511766    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:17.644185 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:17.644151    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:30.764510 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:30.764471    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:35.890246 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890208    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"]
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890572    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4d8053a-ccbb-4802-81a6-49a0678fb7b9" containerName="payload-pre-processing"
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890583    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d8053a-ccbb-4802-81a6-49a0678fb7b9" containerName="payload-pre-processing"
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890594    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e107042-4440-48ec-a728-15ad60318bc0" containerName="maas-api"
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890600    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e107042-4440-48ec-a728-15ad60318bc0" containerName="maas-api"
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890620    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3ea980e-1782-429d-8714-25ff791a8a8d" containerName="payload-processing"
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890625    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3ea980e-1782-429d-8714-25ff791a8a8d" containerName="payload-processing"
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890634    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" containerName="main"
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890639    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" containerName="main"
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890691    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3ea980e-1782-429d-8714-25ff791a8a8d" containerName="payload-processing"
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890702    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e107042-4440-48ec-a728-15ad60318bc0" containerName="maas-api"
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890709    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca27ba8c-b11d-4496-99e6-bee61f6b8c8e" containerName="main"
Sep 05 06:42:35.890846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.890715    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4d8053a-ccbb-4802-81a6-49a0678fb7b9" containerName="payload-pre-processing"
Sep 05 06:42:35.893755 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.893739    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:35.896165 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.896138    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-eee7e8\"/\"openshift-service-ca.crt\""
Sep 05 06:42:35.897121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.897101    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-eee7e8\"/\"default-dockercfg-gwpvn\""
Sep 05 06:42:35.897121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.897116    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-eee7e8\"/\"noten-model-eee7e8-kserve-self-signed-certs\""
Sep 05 06:42:35.897289 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.897101    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-eee7e8\"/\"kube-root-ca.crt\""
Sep 05 06:42:35.905244 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.905220    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"]
Sep 05 06:42:35.911545 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.911522    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-model-cache\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:35.911655 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.911617    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nszbk\" (UniqueName: \"kubernetes.io/projected/60c668cc-b163-4bc0-88a6-2a0f5068562f-kube-api-access-nszbk\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:35.911707 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.911679    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-tmp-dir\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:35.911752 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.911717    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-home\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:35.911795 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.911768    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c668cc-b163-4bc0-88a6-2a0f5068562f-tls-certs\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:35.911880 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.911865    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-dshm\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:35.992612 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:35.992575    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:36.013274 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.013246    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-model-cache\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.013452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.013286    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nszbk\" (UniqueName: \"kubernetes.io/projected/60c668cc-b163-4bc0-88a6-2a0f5068562f-kube-api-access-nszbk\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.013452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.013332    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-tmp-dir\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.013452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.013364    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-home\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.013452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.013381    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c668cc-b163-4bc0-88a6-2a0f5068562f-tls-certs\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.013452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.013429    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-dshm\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.013741 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.013704    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-model-cache\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.013890 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.013840    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-tmp-dir\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.013995 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.013915    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-home\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.015931 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.015908    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-dshm\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.016197 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.016170    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c668cc-b163-4bc0-88a6-2a0f5068562f-tls-certs\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.021936 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.021915    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nszbk\" (UniqueName: \"kubernetes.io/projected/60c668cc-b163-4bc0-88a6-2a0f5068562f-kube-api-access-nszbk\") pod \"noten-model-eee7e8-kserve-b5cb59ff5-zqpcb\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") " pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.203112 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.203025    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:36.475277 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:36.475252    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"]
Sep 05 06:42:36.477440 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:42:36.477415    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60c668cc_b163_4bc0_88a6_2a0f5068562f.slice/crio-8e0d62856b5d5487ce8b22d34ac7e061fcd3743614963dead64633f29185be57 WatchSource:0}: Error finding container 8e0d62856b5d5487ce8b22d34ac7e061fcd3743614963dead64633f29185be57: Status 404 returned error can't find the container with id 8e0d62856b5d5487ce8b22d34ac7e061fcd3743614963dead64633f29185be57
Sep 05 06:42:37.060397 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:37.060366    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb" event={"ID":"60c668cc-b163-4bc0-88a6-2a0f5068562f","Type":"ContainerStarted","Data":"f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed"}
Sep 05 06:42:37.060753 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:37.060406    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb" event={"ID":"60c668cc-b163-4bc0-88a6-2a0f5068562f","Type":"ContainerStarted","Data":"8e0d62856b5d5487ce8b22d34ac7e061fcd3743614963dead64633f29185be57"}
Sep 05 06:42:37.081480 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:37.081435    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb" podStartSLOduration=2.081419912 podStartE2EDuration="2.081419912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:42:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:42:37.078944761 +0000 UTC m=+1005.446991970" watchObservedRunningTime="2026-09-05 06:42:37.081419912 +0000 UTC m=+1005.449467121"
Sep 05 06:42:37.092781 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:37.092754    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:37.197015 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:37.196983    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:37.293653 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:37.293620    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:46.204138 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:46.204106    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:46.204523 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:46.204155    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:46.216669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:46.216646    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:47.120317 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:47.120286    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:42:47.842463 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:47.842426    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:48.649272 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:48.649240    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:52.979311 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:52.979274    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"]
Sep 05 06:42:53.012052 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:53.012018    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:53.873154 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:53.873120    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"]
Sep 05 06:42:53.895585 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:53.895553    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:54.015243 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:54.015207    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:54.251890 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:54.251844    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:54.363550 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:54.363517    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:55.094094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:55.094057    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:55.354603 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:55.354510    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:55.458528 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:55.458493    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:55.561843 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:55.561804    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:55.662201 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:55.662120    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:56.310830 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:56.310801    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:56.409734 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:56.409701    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:56.510210 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:56.510173    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:58.003942 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:58.003909    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:42:59.442751 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:42:59.442717    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:04.060700 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:04.060661    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:04.125288 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:04.125251    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:04.715524 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:04.715488    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:05.805235 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:05.805188    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"]
Sep 05 06:43:05.805703 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:05.805482    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85" podUID="ba3a1c79-f1ba-4f64-9e91-9c2a66277d95" containerName="maas-api" containerID="cri-o://e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e" gracePeriod=30
Sep 05 06:43:05.840841 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:05.840807    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:05.912329 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:05.912267    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:05.978126 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:05.978099    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:06.044535 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.044496    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:06.046934 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.046917    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:43:06.112921 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.112821    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:06.198986 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.198962    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-maas-api-tls\") pod \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") "
Sep 05 06:43:06.199167 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.198996    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-metrics-certs\") pod \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") "
Sep 05 06:43:06.199167 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.199087    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-openshift-service-ca\") pod \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") "
Sep 05 06:43:06.199167 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.199134    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29fsw\" (UniqueName: \"kubernetes.io/projected/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-kube-api-access-29fsw\") pod \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\" (UID: \"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95\") "
Sep 05 06:43:06.199486 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.199461    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ba3a1c79-f1ba-4f64-9e91-9c2a66277d95" (UID: "ba3a1c79-f1ba-4f64-9e91-9c2a66277d95"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:43:06.201304 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.201271    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-kube-api-access-29fsw" (OuterVolumeSpecName: "kube-api-access-29fsw") pod "ba3a1c79-f1ba-4f64-9e91-9c2a66277d95" (UID: "ba3a1c79-f1ba-4f64-9e91-9c2a66277d95"). InnerVolumeSpecName "kube-api-access-29fsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:43:06.201304 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.201283    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ba3a1c79-f1ba-4f64-9e91-9c2a66277d95" (UID: "ba3a1c79-f1ba-4f64-9e91-9c2a66277d95"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:43:06.201452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.201360    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ba3a1c79-f1ba-4f64-9e91-9c2a66277d95" (UID: "ba3a1c79-f1ba-4f64-9e91-9c2a66277d95"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:43:06.207834 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.207810    2573 generic.go:356] "Generic (PLEG): container finished" podID="ba3a1c79-f1ba-4f64-9e91-9c2a66277d95" containerID="e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e" exitCode=0
Sep 05 06:43:06.208005 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.207869    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"
Sep 05 06:43:06.208005 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.207922    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85" event={"ID":"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95","Type":"ContainerDied","Data":"e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e"}
Sep 05 06:43:06.208005 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.207972    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85" event={"ID":"ba3a1c79-f1ba-4f64-9e91-9c2a66277d95","Type":"ContainerDied","Data":"c67af4436896057f492452c589bd7ebe4f553293780dbbb29e28b075b670e17f"}
Sep 05 06:43:06.208005 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.208001    2573 scope.go:117] "RemoveContainer" containerID="e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e"
Sep 05 06:43:06.224464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.224442    2573 scope.go:117] "RemoveContainer" containerID="e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e"
Sep 05 06:43:06.224923 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:06.224898    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e\": container with ID starting with e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e not found: ID does not exist" containerID="e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e"
Sep 05 06:43:06.225019 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.224935    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e"} err="failed to get container status \"e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e\": rpc error: code = NotFound desc = could not find container \"e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e\": container with ID starting with e0443a5679c3aede3eb9a53929f021da93c1b12b3d5449cd8e69284b5461ad0e not found: ID does not exist"
Sep 05 06:43:06.235639 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.235606    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"]
Sep 05 06:43:06.239439 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.239415    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45e697-79c967cd49-ltk85"]
Sep 05 06:43:06.300769 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.300737    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:06.300769 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.300765    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29fsw\" (UniqueName: \"kubernetes.io/projected/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-kube-api-access-29fsw\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:06.301044 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.300781    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:06.301044 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.300794    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:06.395668 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:06.395602    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:07.998738 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:07.998680    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd" podUID="cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" containerName="main" containerID="cri-o://89358aa8b3d6d4ea9ec98e5bd6d421c492fc5e46edb043128ba7717b67885486" gracePeriod=45
Sep 05 06:43:08.156019 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.155993    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba3a1c79-f1ba-4f64-9e91-9c2a66277d95" path="/var/lib/kubelet/pods/ba3a1c79-f1ba-4f64-9e91-9c2a66277d95/volumes"
Sep 05 06:43:08.226192 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.226162    2573 generic.go:356] "Generic (PLEG): container finished" podID="cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" containerID="89358aa8b3d6d4ea9ec98e5bd6d421c492fc5e46edb043128ba7717b67885486" exitCode=0
Sep 05 06:43:08.226335 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.226229    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd" event={"ID":"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf","Type":"ContainerDied","Data":"89358aa8b3d6d4ea9ec98e5bd6d421c492fc5e46edb043128ba7717b67885486"}
Sep 05 06:43:08.249533 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.249475    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:43:08.419694 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.419658    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tls-certs\") pod \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") "
Sep 05 06:43:08.419694 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.419703    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfmf6\" (UniqueName: \"kubernetes.io/projected/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-kube-api-access-sfmf6\") pod \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") "
Sep 05 06:43:08.419974 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.419722    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-dshm\") pod \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") "
Sep 05 06:43:08.419974 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.419863    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-home\") pod \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") "
Sep 05 06:43:08.419974 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.419945    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tmp-dir\") pod \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") "
Sep 05 06:43:08.420153 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.419978    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-model-cache\") pod \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\" (UID: \"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf\") "
Sep 05 06:43:08.420153 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.420118    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-home" (OuterVolumeSpecName: "home") pod "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" (UID: "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:08.420264 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.420206    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" (UID: "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:08.420364 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.420339    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-model-cache" (OuterVolumeSpecName: "model-cache") pod "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" (UID: "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:08.420423 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.420349    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-home\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:08.420423 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.420391    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tmp-dir\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:08.421941 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.421917    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" (UID: "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:43:08.421941 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.421937    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-dshm" (OuterVolumeSpecName: "dshm") pod "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" (UID: "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:08.422078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.421947    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-kube-api-access-sfmf6" (OuterVolumeSpecName: "kube-api-access-sfmf6") pod "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" (UID: "cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf"). InnerVolumeSpecName "kube-api-access-sfmf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:43:08.521516 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.521443    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-tls-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:08.521516 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.521466    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfmf6\" (UniqueName: \"kubernetes.io/projected/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-kube-api-access-sfmf6\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:08.521516 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.521474    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-dshm\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:08.521516 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.521484    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf-model-cache\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:08.892789 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:08.892710    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb" podUID="60c668cc-b163-4bc0-88a6-2a0f5068562f" containerName="main" containerID="cri-o://f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed" gracePeriod=45
Sep 05 06:43:09.131121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.131096    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:43:09.227265 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227185    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-tmp-dir\") pod \"60c668cc-b163-4bc0-88a6-2a0f5068562f\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") "
Sep 05 06:43:09.227265 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227236    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nszbk\" (UniqueName: \"kubernetes.io/projected/60c668cc-b163-4bc0-88a6-2a0f5068562f-kube-api-access-nszbk\") pod \"60c668cc-b163-4bc0-88a6-2a0f5068562f\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") "
Sep 05 06:43:09.227469 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227283    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c668cc-b163-4bc0-88a6-2a0f5068562f-tls-certs\") pod \"60c668cc-b163-4bc0-88a6-2a0f5068562f\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") "
Sep 05 06:43:09.227469 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227356    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-home\") pod \"60c668cc-b163-4bc0-88a6-2a0f5068562f\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") "
Sep 05 06:43:09.227469 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227385    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-dshm\") pod \"60c668cc-b163-4bc0-88a6-2a0f5068562f\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") "
Sep 05 06:43:09.227469 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227458    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-model-cache\") pod \"60c668cc-b163-4bc0-88a6-2a0f5068562f\" (UID: \"60c668cc-b163-4bc0-88a6-2a0f5068562f\") "
Sep 05 06:43:09.227693 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227471    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60c668cc-b163-4bc0-88a6-2a0f5068562f" (UID: "60c668cc-b163-4bc0-88a6-2a0f5068562f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:09.227693 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227587    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-home" (OuterVolumeSpecName: "home") pod "60c668cc-b163-4bc0-88a6-2a0f5068562f" (UID: "60c668cc-b163-4bc0-88a6-2a0f5068562f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:09.227806 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227755    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-model-cache" (OuterVolumeSpecName: "model-cache") pod "60c668cc-b163-4bc0-88a6-2a0f5068562f" (UID: "60c668cc-b163-4bc0-88a6-2a0f5068562f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:09.227899 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227882    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-home\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:09.227953 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227906    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-model-cache\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:09.227953 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.227921    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-tmp-dir\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:09.229611 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.229589    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60c668cc-b163-4bc0-88a6-2a0f5068562f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60c668cc-b163-4bc0-88a6-2a0f5068562f" (UID: "60c668cc-b163-4bc0-88a6-2a0f5068562f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:43:09.229734 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.229593    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-dshm" (OuterVolumeSpecName: "dshm") pod "60c668cc-b163-4bc0-88a6-2a0f5068562f" (UID: "60c668cc-b163-4bc0-88a6-2a0f5068562f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:09.229734 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.229704    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60c668cc-b163-4bc0-88a6-2a0f5068562f-kube-api-access-nszbk" (OuterVolumeSpecName: "kube-api-access-nszbk") pod "60c668cc-b163-4bc0-88a6-2a0f5068562f" (UID: "60c668cc-b163-4bc0-88a6-2a0f5068562f"). InnerVolumeSpecName "kube-api-access-nszbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:43:09.233639 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.233612    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"
Sep 05 06:43:09.233750 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.233647    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd" event={"ID":"cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf","Type":"ContainerDied","Data":"604bbfa53ab8d7df1dc53e2435afc528874dd44e4a60f9ab5c711835de185b1c"}
Sep 05 06:43:09.233750 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.233706    2573 scope.go:117] "RemoveContainer" containerID="89358aa8b3d6d4ea9ec98e5bd6d421c492fc5e46edb043128ba7717b67885486"
Sep 05 06:43:09.235214 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.235193    2573 generic.go:356] "Generic (PLEG): container finished" podID="60c668cc-b163-4bc0-88a6-2a0f5068562f" containerID="f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed" exitCode=0
Sep 05 06:43:09.235326 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.235222    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb" event={"ID":"60c668cc-b163-4bc0-88a6-2a0f5068562f","Type":"ContainerDied","Data":"f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed"}
Sep 05 06:43:09.235326 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.235240    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"
Sep 05 06:43:09.235326 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.235266    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb" event={"ID":"60c668cc-b163-4bc0-88a6-2a0f5068562f","Type":"ContainerDied","Data":"8e0d62856b5d5487ce8b22d34ac7e061fcd3743614963dead64633f29185be57"}
Sep 05 06:43:09.249945 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.249925    2573 scope.go:117] "RemoveContainer" containerID="f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed"
Sep 05 06:43:09.257310 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.257296    2573 scope.go:117] "RemoveContainer" containerID="f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed"
Sep 05 06:43:09.257543 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:09.257525    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed\": container with ID starting with f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed not found: ID does not exist" containerID="f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed"
Sep 05 06:43:09.257585 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.257551    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed"} err="failed to get container status \"f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed\": rpc error: code = NotFound desc = could not find container \"f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed\": container with ID starting with f339f343e55ae186e1d6f60514f48e8a87b3b8fdb0fd429901b7e1b9b53197ed not found: ID does not exist"
Sep 05 06:43:09.262862 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.262833    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"]
Sep 05 06:43:09.270768 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.270748    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-45e697/test-model-45e697-kserve-545757cb4f-mrhrd"]
Sep 05 06:43:09.282917 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.282895    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"]
Sep 05 06:43:09.286816 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.286798    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-eee7e8/noten-model-eee7e8-kserve-b5cb59ff5-zqpcb"]
Sep 05 06:43:09.328527 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.328506    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c668cc-b163-4bc0-88a6-2a0f5068562f-dshm\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:09.328527 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.328526    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nszbk\" (UniqueName: \"kubernetes.io/projected/60c668cc-b163-4bc0-88a6-2a0f5068562f-kube-api-access-nszbk\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:09.328638 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:09.328537    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c668cc-b163-4bc0-88a6-2a0f5068562f-tls-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:10.156162 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:10.156133    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60c668cc-b163-4bc0-88a6-2a0f5068562f" path="/var/lib/kubelet/pods/60c668cc-b163-4bc0-88a6-2a0f5068562f/volumes"
Sep 05 06:43:10.156532 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:10.156523    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" path="/var/lib/kubelet/pods/cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf/volumes"
Sep 05 06:43:10.941169 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:10.941134    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:11.041879 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:11.041823    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:11.163206 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:11.163174    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:11.463286 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:11.463250    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:12.760704 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:12.760673    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:13.154258 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:13.154168    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:13.255289 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:13.255253    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:13.415268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:13.415174    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:13.848772 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:13.848737    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:14.267710 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:14.267672    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:14.358607 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:14.358575    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:15.417983 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.417951    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"]
Sep 05 06:43:15.419066 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.419038    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" containerName="main"
Sep 05 06:43:15.419066 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.419066    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" containerName="main"
Sep 05 06:43:15.419215 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.419082    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba3a1c79-f1ba-4f64-9e91-9c2a66277d95" containerName="maas-api"
Sep 05 06:43:15.419215 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.419091    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba3a1c79-f1ba-4f64-9e91-9c2a66277d95" containerName="maas-api"
Sep 05 06:43:15.419215 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.419125    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60c668cc-b163-4bc0-88a6-2a0f5068562f" containerName="main"
Sep 05 06:43:15.419215 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.419133    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="60c668cc-b163-4bc0-88a6-2a0f5068562f" containerName="main"
Sep 05 06:43:15.419336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.419241    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="60c668cc-b163-4bc0-88a6-2a0f5068562f" containerName="main"
Sep 05 06:43:15.419336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.419254    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba3a1c79-f1ba-4f64-9e91-9c2a66277d95" containerName="maas-api"
Sep 05 06:43:15.419336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.419267    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0ae0a7-ccdd-43b2-b841-efe4c144dcbf" containerName="main"
Sep 05 06:43:15.424324 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.424295    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.427102 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.427081    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-810a2bb6-serving-cert\""
Sep 05 06:43:15.431616 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.431594    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"]
Sep 05 06:43:15.462470 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.462441    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:15.477910 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.477887    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.478014 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.477926    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msfd8\" (UniqueName: \"kubernetes.io/projected/33deec3c-3fc0-45cb-aea8-1165983f630f-kube-api-access-msfd8\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.478014 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.477957    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-metrics-certs\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.478088 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.478042    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33deec3c-3fc0-45cb-aea8-1165983f630f-openshift-service-ca\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.564414 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.564383    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:15.578819 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.578796    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.578943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.578836    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msfd8\" (UniqueName: \"kubernetes.io/projected/33deec3c-3fc0-45cb-aea8-1165983f630f-kube-api-access-msfd8\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.578943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.578875    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-metrics-certs\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.578943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.578912    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33deec3c-3fc0-45cb-aea8-1165983f630f-openshift-service-ca\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.579108 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:15.578959    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-serving-cert: secret "maas-api-e2e-mt-810a2bb6-serving-cert" not found
Sep 05 06:43:15.579108 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:15.579044    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls podName:33deec3c-3fc0-45cb-aea8-1165983f630f nodeName:}" failed. No retries permitted until 2026-09-05 06:43:16.079023314 +0000 UTC m=+1044.447070505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls") pod "maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx" (UID: "33deec3c-3fc0-45cb-aea8-1165983f630f") : secret "maas-api-e2e-mt-810a2bb6-serving-cert" not found
Sep 05 06:43:15.579593 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.579574    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33deec3c-3fc0-45cb-aea8-1165983f630f-openshift-service-ca\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.581322 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.581301    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-metrics-certs\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.587804 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.587784    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msfd8\" (UniqueName: \"kubernetes.io/projected/33deec3c-3fc0-45cb-aea8-1165983f630f-kube-api-access-msfd8\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:15.998808 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:15.998777    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:16.082318 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.082288    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:16.082483 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:16.082461    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-serving-cert: secret "maas-api-e2e-mt-810a2bb6-serving-cert" not found
Sep 05 06:43:16.082552 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:16.082541    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls podName:33deec3c-3fc0-45cb-aea8-1165983f630f nodeName:}" failed. No retries permitted until 2026-09-05 06:43:17.082521028 +0000 UTC m=+1045.450568222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls") pod "maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx" (UID: "33deec3c-3fc0-45cb-aea8-1165983f630f") : secret "maas-api-e2e-mt-810a2bb6-serving-cert" not found
Sep 05 06:43:16.098166 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.098139    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:16.125032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.125003    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"]
Sep 05 06:43:16.128912 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.128892    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"]
Sep 05 06:43:16.129053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.129026    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:16.131901 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.131864    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-810a2bb6\""
Sep 05 06:43:16.132120 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.132103    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:16.139758 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.139734    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"]
Sep 05 06:43:16.142810 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.142788    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"]
Sep 05 06:43:16.183592 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.183559    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsnkz\" (UniqueName: \"kubernetes.io/projected/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-kube-api-access-dsnkz\") pod \"payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg\" (UID: \"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:16.183786 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.183602    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn5w7\" (UniqueName: \"kubernetes.io/projected/3138f393-301c-4fef-bdb9-0670098769a7-kube-api-access-sn5w7\") pod \"payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr\" (UID: \"3138f393-301c-4fef-bdb9-0670098769a7\") " pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:16.183786 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.183726    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg\" (UID: \"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:16.183786 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.183759    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3138f393-301c-4fef-bdb9-0670098769a7-plugins-config-volume\") pod \"payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr\" (UID: \"3138f393-301c-4fef-bdb9-0670098769a7\") " pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:16.285032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.284960    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg\" (UID: \"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:16.285032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.284996    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3138f393-301c-4fef-bdb9-0670098769a7-plugins-config-volume\") pod \"payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr\" (UID: \"3138f393-301c-4fef-bdb9-0670098769a7\") " pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:16.285032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.285022    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsnkz\" (UniqueName: \"kubernetes.io/projected/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-kube-api-access-dsnkz\") pod \"payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg\" (UID: \"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:16.285310 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.285057    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn5w7\" (UniqueName: \"kubernetes.io/projected/3138f393-301c-4fef-bdb9-0670098769a7-kube-api-access-sn5w7\") pod \"payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr\" (UID: \"3138f393-301c-4fef-bdb9-0670098769a7\") " pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:16.285687 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.285670    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3138f393-301c-4fef-bdb9-0670098769a7-plugins-config-volume\") pod \"payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr\" (UID: \"3138f393-301c-4fef-bdb9-0670098769a7\") " pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:16.285736 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.285687    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg\" (UID: \"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:16.297138 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.297110    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn5w7\" (UniqueName: \"kubernetes.io/projected/3138f393-301c-4fef-bdb9-0670098769a7-kube-api-access-sn5w7\") pod \"payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr\" (UID: \"3138f393-301c-4fef-bdb9-0670098769a7\") " pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:16.297244 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.297194    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsnkz\" (UniqueName: \"kubernetes.io/projected/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-kube-api-access-dsnkz\") pod \"payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg\" (UID: \"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:16.441987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.441959    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:16.449954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.449930    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:16.680259 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.680166    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:16.724407 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.724383    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"]
Sep 05 06:43:16.727458 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:43:16.727425    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae2f4a2_83aa_4bb9_b2ca_a23d360e394f.slice/crio-e498374ae4a604f060d8ded7f2f09bdc85d298fd6ddc77fa685096dbc3ad12fd WatchSource:0}: Error finding container e498374ae4a604f060d8ded7f2f09bdc85d298fd6ddc77fa685096dbc3ad12fd: Status 404 returned error can't find the container with id e498374ae4a604f060d8ded7f2f09bdc85d298fd6ddc77fa685096dbc3ad12fd
Sep 05 06:43:16.729419 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:16.729385    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"]
Sep 05 06:43:16.731053 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:43:16.731032    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3138f393_301c_4fef_bdb9_0670098769a7.slice/crio-077ff791b7c973f1f88a377ebc08cd0a10e039c4e4dc2a4d502994fb33ddbb78 WatchSource:0}: Error finding container 077ff791b7c973f1f88a377ebc08cd0a10e039c4e4dc2a4d502994fb33ddbb78: Status 404 returned error can't find the container with id 077ff791b7c973f1f88a377ebc08cd0a10e039c4e4dc2a4d502994fb33ddbb78
Sep 05 06:43:17.093326 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.093292    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:17.095647 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.095625    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls\") pod \"maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:17.239118 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.239071    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:17.278790 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.278744    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr" event={"ID":"3138f393-301c-4fef-bdb9-0670098769a7","Type":"ContainerStarted","Data":"f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160"}
Sep 05 06:43:17.278968 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.278833    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr" event={"ID":"3138f393-301c-4fef-bdb9-0670098769a7","Type":"ContainerStarted","Data":"077ff791b7c973f1f88a377ebc08cd0a10e039c4e4dc2a4d502994fb33ddbb78"}
Sep 05 06:43:17.279803 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.279779    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:17.287787 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.287288    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg" event={"ID":"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f","Type":"ContainerStarted","Data":"ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b"}
Sep 05 06:43:17.287787 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.287339    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg" event={"ID":"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f","Type":"ContainerStarted","Data":"e498374ae4a604f060d8ded7f2f09bdc85d298fd6ddc77fa685096dbc3ad12fd"}
Sep 05 06:43:17.288024 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.287895    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:17.302202 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.302157    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr" podStartSLOduration=1.302137691 podStartE2EDuration="1.302137691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:43:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:43:17.30101384 +0000 UTC m=+1045.669061053" watchObservedRunningTime="2026-09-05 06:43:17.302137691 +0000 UTC m=+1045.670184903"
Sep 05 06:43:17.319878 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.319528    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg" podStartSLOduration=1.319505119 podStartE2EDuration="1.319505119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:43:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:43:17.31821309 +0000 UTC m=+1045.686260302" watchObservedRunningTime="2026-09-05 06:43:17.319505119 +0000 UTC m=+1045.687552331"
Sep 05 06:43:17.544452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.544419    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"]
Sep 05 06:43:17.546801 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:43:17.546773    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33deec3c_3fc0_45cb_aea8_1165983f630f.slice/crio-00c3002896e857f609b5473db6a439792eb66f98b5b82e64b4d084e24fd4f18d WatchSource:0}: Error finding container 00c3002896e857f609b5473db6a439792eb66f98b5b82e64b4d084e24fd4f18d: Status 404 returned error can't find the container with id 00c3002896e857f609b5473db6a439792eb66f98b5b82e64b4d084e24fd4f18d
Sep 05 06:43:17.548609 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:17.548592    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 06:43:18.294148 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:18.294107    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx" event={"ID":"33deec3c-3fc0-45cb-aea8-1165983f630f","Type":"ContainerStarted","Data":"f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73"}
Sep 05 06:43:18.294341 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:18.294165    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx" event={"ID":"33deec3c-3fc0-45cb-aea8-1165983f630f","Type":"ContainerStarted","Data":"00c3002896e857f609b5473db6a439792eb66f98b5b82e64b4d084e24fd4f18d"}
Sep 05 06:43:18.294341 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:18.294252    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:18.313809 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:18.313745    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx" podStartSLOduration=2.794490993 podStartE2EDuration="3.313730501s" totalImagesPullingTime="519.239508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:43:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:43:18.312129233 +0000 UTC m=+1046.680176445" watchObservedRunningTime="2026-09-05 06:43:18.313730501 +0000 UTC m=+1046.681777710"
Sep 05 06:43:23.871671 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:23.871632    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:24.304923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:24.304891    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:24.557015 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:24.556936    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"]
Sep 05 06:43:24.557239 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:24.557214    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66" podUID="26f67427-fe22-4937-b82a-5bcd687049db" containerName="maas-api" containerID="cri-o://ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61" gracePeriod=30
Sep 05 06:43:24.589112 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:24.589082    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:24.672818 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:24.672783    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:24.735015 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:24.734986    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:24.810534 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:24.810457    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:24.869223 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:24.869184    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:25.001508 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.001425    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:43:25.066278 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.066208    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26f67427-fe22-4937-b82a-5bcd687049db-openshift-service-ca\") pod \"26f67427-fe22-4937-b82a-5bcd687049db\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") "
Sep 05 06:43:25.066278 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.066254    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qc9x\" (UniqueName: \"kubernetes.io/projected/26f67427-fe22-4937-b82a-5bcd687049db-kube-api-access-2qc9x\") pod \"26f67427-fe22-4937-b82a-5bcd687049db\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") "
Sep 05 06:43:25.066500 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.066332    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-metrics-certs\") pod \"26f67427-fe22-4937-b82a-5bcd687049db\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") "
Sep 05 06:43:25.066500 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.066357    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-maas-api-tls\") pod \"26f67427-fe22-4937-b82a-5bcd687049db\" (UID: \"26f67427-fe22-4937-b82a-5bcd687049db\") "
Sep 05 06:43:25.066656 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.066624    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26f67427-fe22-4937-b82a-5bcd687049db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "26f67427-fe22-4937-b82a-5bcd687049db" (UID: "26f67427-fe22-4937-b82a-5bcd687049db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:43:25.066865 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.066835    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26f67427-fe22-4937-b82a-5bcd687049db-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:25.068427 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.068392    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26f67427-fe22-4937-b82a-5bcd687049db-kube-api-access-2qc9x" (OuterVolumeSpecName: "kube-api-access-2qc9x") pod "26f67427-fe22-4937-b82a-5bcd687049db" (UID: "26f67427-fe22-4937-b82a-5bcd687049db"). InnerVolumeSpecName "kube-api-access-2qc9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:43:25.068530 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.068434    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "26f67427-fe22-4937-b82a-5bcd687049db" (UID: "26f67427-fe22-4937-b82a-5bcd687049db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:43:25.068530 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.068477    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "26f67427-fe22-4937-b82a-5bcd687049db" (UID: "26f67427-fe22-4937-b82a-5bcd687049db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:43:25.167713 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.167680    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qc9x\" (UniqueName: \"kubernetes.io/projected/26f67427-fe22-4937-b82a-5bcd687049db-kube-api-access-2qc9x\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:25.167713 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.167709    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:25.167713 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.167719    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26f67427-fe22-4937-b82a-5bcd687049db-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:25.188362 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.188305    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:25.329370 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.329277    2573 generic.go:356] "Generic (PLEG): container finished" podID="26f67427-fe22-4937-b82a-5bcd687049db" containerID="ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61" exitCode=0
Sep 05 06:43:25.329370 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.329325    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"
Sep 05 06:43:25.329568 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.329362    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66" event={"ID":"26f67427-fe22-4937-b82a-5bcd687049db","Type":"ContainerDied","Data":"ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61"}
Sep 05 06:43:25.329568 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.329416    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66" event={"ID":"26f67427-fe22-4937-b82a-5bcd687049db","Type":"ContainerDied","Data":"d2d3d86c1f6949d60160902d7b33259b31626b04cb88019208e8104c501a0f59"}
Sep 05 06:43:25.329568 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.329441    2573 scope.go:117] "RemoveContainer" containerID="ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61"
Sep 05 06:43:25.338561 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.338545    2573 scope.go:117] "RemoveContainer" containerID="ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61"
Sep 05 06:43:25.338835 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:25.338819    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61\": container with ID starting with ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61 not found: ID does not exist" containerID="ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61"
Sep 05 06:43:25.338910 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.338845    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61"} err="failed to get container status \"ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61\": rpc error: code = NotFound desc = could not find container \"ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61\": container with ID starting with ddd766139b323340ba6c706ad57fe4503b67d637cf584d5d4b186186ad444c61 not found: ID does not exist"
Sep 05 06:43:25.352094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.352067    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"]
Sep 05 06:43:25.357207 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:25.357182    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7080a4-5d5f74c6cd-r7v66"]
Sep 05 06:43:26.155968 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:26.155934    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26f67427-fe22-4937-b82a-5bcd687049db" path="/var/lib/kubelet/pods/26f67427-fe22-4937-b82a-5bcd687049db/volumes"
Sep 05 06:43:29.301225 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:29.301172    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:29.301596 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:29.301266    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:29.409140 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:29.409092    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:29.503368 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:29.503325    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:31.413006 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:31.412964    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:31.494800 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:31.494765    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:31.693119 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:31.693005    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:32.267315 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:32.267256    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:32.590398 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:32.590292    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:33.051483 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:33.051432    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:35.379042 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:35.379013    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:35.471143 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:35.471103    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:36.470961 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:36.470926    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:37.316041 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.316004    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"]
Sep 05 06:43:37.316410 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.316395    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26f67427-fe22-4937-b82a-5bcd687049db" containerName="maas-api"
Sep 05 06:43:37.316480 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.316413    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="26f67427-fe22-4937-b82a-5bcd687049db" containerName="maas-api"
Sep 05 06:43:37.316534 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.316494    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="26f67427-fe22-4937-b82a-5bcd687049db" containerName="maas-api"
Sep 05 06:43:37.321210 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.321179    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.323987 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.323966    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-183ae6-serving-cert\""
Sep 05 06:43:37.331143 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.331113    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"]
Sep 05 06:43:37.353190 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.353157    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:37.453517 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.453480    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:37.478662 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.478627    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62500340-a383-45b4-8dbb-e3f9e02e292f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.479099 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.478683    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.479099 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.478771    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5llr\" (UniqueName: \"kubernetes.io/projected/62500340-a383-45b4-8dbb-e3f9e02e292f-kube-api-access-w5llr\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.479099 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.478806    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.554000 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.553961    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:37.580144 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.580046    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5llr\" (UniqueName: \"kubernetes.io/projected/62500340-a383-45b4-8dbb-e3f9e02e292f-kube-api-access-w5llr\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.580144 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.580096    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.580144 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.580137    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62500340-a383-45b4-8dbb-e3f9e02e292f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.580403 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.580204    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.580403 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:37.580337    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-serving-cert: secret "maas-api-e2e-shared-a-w3-183ae6-serving-cert" not found
Sep 05 06:43:37.580489 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:37.580413    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-maas-api-tls podName:62500340-a383-45b4-8dbb-e3f9e02e292f nodeName:}" failed. No retries permitted until 2026-09-05 06:43:38.080382338 +0000 UTC m=+1066.448429540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-maas-api-tls") pod "maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w" (UID: "62500340-a383-45b4-8dbb-e3f9e02e292f") : secret "maas-api-e2e-shared-a-w3-183ae6-serving-cert" not found
Sep 05 06:43:37.580970 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.580941    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62500340-a383-45b4-8dbb-e3f9e02e292f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.583380 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.583357    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.591203 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.591179    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5llr\" (UniqueName: \"kubernetes.io/projected/62500340-a383-45b4-8dbb-e3f9e02e292f-kube-api-access-w5llr\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:37.820567 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:37.820527    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:38.044816 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:38.044781    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:38.085737 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:38.085677    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:38.088179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:38.088156    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:38.143914 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:38.143885    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:38.233960 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:38.233924    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:38.474044 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:38.473956    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:38.509941 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:38.509916    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"]
Sep 05 06:43:38.512088 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:43:38.512064    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62500340_a383_45b4_8dbb_e3f9e02e292f.slice/crio-f45068577c78fea34282fca50f04e0b34f26206c6402d0bceb0e3a96e334d4ca WatchSource:0}: Error finding container f45068577c78fea34282fca50f04e0b34f26206c6402d0bceb0e3a96e334d4ca: Status 404 returned error can't find the container with id f45068577c78fea34282fca50f04e0b34f26206c6402d0bceb0e3a96e334d4ca
Sep 05 06:43:39.267669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:39.267611    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:39.400814 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:39.400731    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w" event={"ID":"62500340-a383-45b4-8dbb-e3f9e02e292f","Type":"ContainerStarted","Data":"a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742"}
Sep 05 06:43:39.400814 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:39.400795    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:39.401009 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:39.400820    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w" event={"ID":"62500340-a383-45b4-8dbb-e3f9e02e292f","Type":"ContainerStarted","Data":"f45068577c78fea34282fca50f04e0b34f26206c6402d0bceb0e3a96e334d4ca"}
Sep 05 06:43:39.423978 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:39.423950    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:39.432140 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:39.432099    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w" podStartSLOduration=1.903471754 podStartE2EDuration="2.432084047s" totalImagesPullingTime="528.612293ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:43:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:43:39.430097666 +0000 UTC m=+1067.798144876" watchObservedRunningTime="2026-09-05 06:43:39.432084047 +0000 UTC m=+1067.800131253"
Sep 05 06:43:39.899369 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:39.899333    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:39.999030 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:39.998998    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:40.343846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:40.343805    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:45.410452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:45.410369    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:43:46.656890 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:46.656839    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:46.757214 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:46.757178    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:47.132058 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:47.132013    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:57.789671 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:57.789638    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"]
Sep 05 06:43:57.790123 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:57.789845    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx" podUID="33deec3c-3fc0-45cb-aea8-1165983f630f" containerName="maas-api" containerID="cri-o://f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73" gracePeriod=30
Sep 05 06:43:57.813194 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:57.813149    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:57.842722 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:57.842686    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"]
Sep 05 06:43:57.843051 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:57.843022    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr" podUID="3138f393-301c-4fef-bdb9-0670098769a7" containerName="payload-processing" containerID="cri-o://f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160" gracePeriod=30
Sep 05 06:43:57.850813 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:57.850788    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"]
Sep 05 06:43:57.851055 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:57.851030    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg" podUID="6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f" containerName="payload-pre-processing" containerID="cri-o://ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b" gracePeriod=30
Sep 05 06:43:57.888977 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:57.888949    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:57.958740 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:57.958703    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:58.038090 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.038056    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:58.103347 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.103312    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:58.112127 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.112105    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:58.127744 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.127720    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:58.140057 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.140035    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:58.163813 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.163782    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-plugins-config-volume\") pod \"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f\" (UID: \"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f\") "
Sep 05 06:43:58.163983 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.163828    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsnkz\" (UniqueName: \"kubernetes.io/projected/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-kube-api-access-dsnkz\") pod \"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f\" (UID: \"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f\") "
Sep 05 06:43:58.163983 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.163885    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33deec3c-3fc0-45cb-aea8-1165983f630f-openshift-service-ca\") pod \"33deec3c-3fc0-45cb-aea8-1165983f630f\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") "
Sep 05 06:43:58.163983 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.163925    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn5w7\" (UniqueName: \"kubernetes.io/projected/3138f393-301c-4fef-bdb9-0670098769a7-kube-api-access-sn5w7\") pod \"3138f393-301c-4fef-bdb9-0670098769a7\" (UID: \"3138f393-301c-4fef-bdb9-0670098769a7\") "
Sep 05 06:43:58.164157 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.163987    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msfd8\" (UniqueName: \"kubernetes.io/projected/33deec3c-3fc0-45cb-aea8-1165983f630f-kube-api-access-msfd8\") pod \"33deec3c-3fc0-45cb-aea8-1165983f630f\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") "
Sep 05 06:43:58.164157 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.164048    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-metrics-certs\") pod \"33deec3c-3fc0-45cb-aea8-1165983f630f\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") "
Sep 05 06:43:58.164157 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.164077    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3138f393-301c-4fef-bdb9-0670098769a7-plugins-config-volume\") pod \"3138f393-301c-4fef-bdb9-0670098769a7\" (UID: \"3138f393-301c-4fef-bdb9-0670098769a7\") "
Sep 05 06:43:58.164157 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.164116    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls\") pod \"33deec3c-3fc0-45cb-aea8-1165983f630f\" (UID: \"33deec3c-3fc0-45cb-aea8-1165983f630f\") "
Sep 05 06:43:58.164366 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.164237    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f" (UID: "6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:43:58.164366 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.164264    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33deec3c-3fc0-45cb-aea8-1165983f630f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33deec3c-3fc0-45cb-aea8-1165983f630f" (UID: "33deec3c-3fc0-45cb-aea8-1165983f630f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:43:58.164479 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.164407    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:58.164479 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.164427    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33deec3c-3fc0-45cb-aea8-1165983f630f-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:58.164831 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.164658    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3138f393-301c-4fef-bdb9-0670098769a7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3138f393-301c-4fef-bdb9-0670098769a7" (UID: "3138f393-301c-4fef-bdb9-0670098769a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:43:58.166566 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.166533    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-kube-api-access-dsnkz" (OuterVolumeSpecName: "kube-api-access-dsnkz") pod "6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f" (UID: "6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f"). InnerVolumeSpecName "kube-api-access-dsnkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:43:58.167448 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.167420    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3138f393-301c-4fef-bdb9-0670098769a7-kube-api-access-sn5w7" (OuterVolumeSpecName: "kube-api-access-sn5w7") pod "3138f393-301c-4fef-bdb9-0670098769a7" (UID: "3138f393-301c-4fef-bdb9-0670098769a7"). InnerVolumeSpecName "kube-api-access-sn5w7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:43:58.167695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.167521    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "33deec3c-3fc0-45cb-aea8-1165983f630f" (UID: "33deec3c-3fc0-45cb-aea8-1165983f630f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:43:58.167695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.167563    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33deec3c-3fc0-45cb-aea8-1165983f630f-kube-api-access-msfd8" (OuterVolumeSpecName: "kube-api-access-msfd8") pod "33deec3c-3fc0-45cb-aea8-1165983f630f" (UID: "33deec3c-3fc0-45cb-aea8-1165983f630f"). InnerVolumeSpecName "kube-api-access-msfd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:43:58.167837 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.167808    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33deec3c-3fc0-45cb-aea8-1165983f630f" (UID: "33deec3c-3fc0-45cb-aea8-1165983f630f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:43:58.265681 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.265653    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:58.265681 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.265676    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsnkz\" (UniqueName: \"kubernetes.io/projected/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f-kube-api-access-dsnkz\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:58.265681 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.265687    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn5w7\" (UniqueName: \"kubernetes.io/projected/3138f393-301c-4fef-bdb9-0670098769a7-kube-api-access-sn5w7\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:58.265923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.265697    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msfd8\" (UniqueName: \"kubernetes.io/projected/33deec3c-3fc0-45cb-aea8-1165983f630f-kube-api-access-msfd8\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:58.265923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.265707    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33deec3c-3fc0-45cb-aea8-1165983f630f-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:58.265923 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.265716    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3138f393-301c-4fef-bdb9-0670098769a7-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:43:58.408079 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.407992    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:58.495208 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.495174    2573 generic.go:356] "Generic (PLEG): container finished" podID="3138f393-301c-4fef-bdb9-0670098769a7" containerID="f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160" exitCode=0
Sep 05 06:43:58.495408 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.495218    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"
Sep 05 06:43:58.495408 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.495251    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr" event={"ID":"3138f393-301c-4fef-bdb9-0670098769a7","Type":"ContainerDied","Data":"f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160"}
Sep 05 06:43:58.495408 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.495307    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr" event={"ID":"3138f393-301c-4fef-bdb9-0670098769a7","Type":"ContainerDied","Data":"077ff791b7c973f1f88a377ebc08cd0a10e039c4e4dc2a4d502994fb33ddbb78"}
Sep 05 06:43:58.495408 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.495338    2573 scope.go:117] "RemoveContainer" containerID="f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160"
Sep 05 06:43:58.496585 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.496561    2573 generic.go:356] "Generic (PLEG): container finished" podID="33deec3c-3fc0-45cb-aea8-1165983f630f" containerID="f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73" exitCode=0
Sep 05 06:43:58.496786 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.496606    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"
Sep 05 06:43:58.496786 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.496620    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx" event={"ID":"33deec3c-3fc0-45cb-aea8-1165983f630f","Type":"ContainerDied","Data":"f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73"}
Sep 05 06:43:58.496786 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.496677    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx" event={"ID":"33deec3c-3fc0-45cb-aea8-1165983f630f","Type":"ContainerDied","Data":"00c3002896e857f609b5473db6a439792eb66f98b5b82e64b4d084e24fd4f18d"}
Sep 05 06:43:58.498162 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.498138    2573 generic.go:356] "Generic (PLEG): container finished" podID="6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f" containerID="ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b" exitCode=0
Sep 05 06:43:58.498259 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.498168    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"
Sep 05 06:43:58.498259 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.498188    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg" event={"ID":"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f","Type":"ContainerDied","Data":"ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b"}
Sep 05 06:43:58.498259 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.498241    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg" event={"ID":"6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f","Type":"ContainerDied","Data":"e498374ae4a604f060d8ded7f2f09bdc85d298fd6ddc77fa685096dbc3ad12fd"}
Sep 05 06:43:58.505131 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.504867    2573 scope.go:117] "RemoveContainer" containerID="f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160"
Sep 05 06:43:58.505224 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:58.505150    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160\": container with ID starting with f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160 not found: ID does not exist" containerID="f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160"
Sep 05 06:43:58.505224 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.505191    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160"} err="failed to get container status \"f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160\": rpc error: code = NotFound desc = could not find container \"f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160\": container with ID starting with f5df4325fe538049522586398065b734c916469c886dbac94bbf4090528f1160 not found: ID does not exist"
Sep 05 06:43:58.505343 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.505223    2573 scope.go:117] "RemoveContainer" containerID="f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73"
Sep 05 06:43:58.507716 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.507692    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:43:58.514235 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.514219    2573 scope.go:117] "RemoveContainer" containerID="f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73"
Sep 05 06:43:58.514503 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:58.514485    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73\": container with ID starting with f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73 not found: ID does not exist" containerID="f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73"
Sep 05 06:43:58.514554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.514509    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73"} err="failed to get container status \"f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73\": rpc error: code = NotFound desc = could not find container \"f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73\": container with ID starting with f716b3d59df076ad12b5b236996504fe16fcf847b0074893fda1130e3185ab73 not found: ID does not exist"
Sep 05 06:43:58.514554 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.514524    2573 scope.go:117] "RemoveContainer" containerID="ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b"
Sep 05 06:43:58.522189 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.522173    2573 scope.go:117] "RemoveContainer" containerID="ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b"
Sep 05 06:43:58.522468 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:43:58.522448    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b\": container with ID starting with ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b not found: ID does not exist" containerID="ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b"
Sep 05 06:43:58.522517 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.522482    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b"} err="failed to get container status \"ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b\": rpc error: code = NotFound desc = could not find container \"ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b\": container with ID starting with ee4a0b9df6eb4ba18777ea298cfcd1f5104ff6e71ba0feb52fd71a9985aad62b not found: ID does not exist"
Sep 05 06:43:58.523314 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.523296    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"]
Sep 05 06:43:58.528011 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.527992    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-810a2bb6-74b78f7bf7-cpzfr"]
Sep 05 06:43:58.542575 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.542555    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"]
Sep 05 06:43:58.549142 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.549119    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-810a2bb6-7c57bb49f9-xgcjx"]
Sep 05 06:43:58.563037 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.563005    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"]
Sep 05 06:43:58.567338 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:43:58.567316    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-810a2bb6-769b497446-28rfg"]
Sep 05 06:44:00.155094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:00.155065    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3138f393-301c-4fef-bdb9-0670098769a7" path="/var/lib/kubelet/pods/3138f393-301c-4fef-bdb9-0670098769a7/volumes"
Sep 05 06:44:00.155559 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:00.155496    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33deec3c-3fc0-45cb-aea8-1165983f630f" path="/var/lib/kubelet/pods/33deec3c-3fc0-45cb-aea8-1165983f630f/volumes"
Sep 05 06:44:00.155816 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:00.155803    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f" path="/var/lib/kubelet/pods/6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f/volumes"
Sep 05 06:44:00.367981 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:00.367954    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:00.678584 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:00.678552    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:01.054414 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:01.054375    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:01.598539 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:01.598507    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:02.348116 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:02.348085    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:02.409802 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:02.409767    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:04.593335 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:04.593301    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:04.741472 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:04.741438    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:05.004217 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.004183    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:05.072022 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.071963    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:05.278504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.278414    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:05.657886 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.657790    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:05.779862 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.779814    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:05.827384 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.827341    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"]
Sep 05 06:44:05.827918 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.827902    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f" containerName="payload-pre-processing"
Sep 05 06:44:05.828010 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.827921    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f" containerName="payload-pre-processing"
Sep 05 06:44:05.828010 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.827948    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33deec3c-3fc0-45cb-aea8-1165983f630f" containerName="maas-api"
Sep 05 06:44:05.828010 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.827956    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="33deec3c-3fc0-45cb-aea8-1165983f630f" containerName="maas-api"
Sep 05 06:44:05.828010 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.827973    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3138f393-301c-4fef-bdb9-0670098769a7" containerName="payload-processing"
Sep 05 06:44:05.828010 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.827982    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3138f393-301c-4fef-bdb9-0670098769a7" containerName="payload-processing"
Sep 05 06:44:05.828170 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.828058    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ae2f4a2-83aa-4bb9-b2ca-a23d360e394f" containerName="payload-pre-processing"
Sep 05 06:44:05.828170 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.828071    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="33deec3c-3fc0-45cb-aea8-1165983f630f" containerName="maas-api"
Sep 05 06:44:05.828170 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.828082    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3138f393-301c-4fef-bdb9-0670098769a7" containerName="payload-processing"
Sep 05 06:44:05.833139 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.833081    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:05.835663 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.835643    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-39047b-serving-cert\""
Sep 05 06:44:05.846196 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.846170    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"]
Sep 05 06:44:05.887126 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.887089    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:05.935276 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.935209    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:05.935276 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.935246    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/823aa414-5ccc-468c-95d7-6207de93926d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:05.935276 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.935268    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz2kq\" (UniqueName: \"kubernetes.io/projected/823aa414-5ccc-468c-95d7-6207de93926d-kube-api-access-bz2kq\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:05.935495 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.935284    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:05.988027 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:05.987992    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:06.036015 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.035973    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:06.036152 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.036128    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/823aa414-5ccc-468c-95d7-6207de93926d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:06.036217 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.036170    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz2kq\" (UniqueName: \"kubernetes.io/projected/823aa414-5ccc-468c-95d7-6207de93926d-kube-api-access-bz2kq\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:06.036303 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.036282    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:06.036748 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.036726    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/823aa414-5ccc-468c-95d7-6207de93926d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:06.038422 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.038394    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:06.038537 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.038519    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:06.044958 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.044937    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz2kq\" (UniqueName: \"kubernetes.io/projected/823aa414-5ccc-468c-95d7-6207de93926d-kube-api-access-bz2kq\") pod \"maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:06.087690 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.087659    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:06.152916 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.152890    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:06.398314 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.398279    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:06.426538 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.426512    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"]
Sep 05 06:44:06.428296 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:44:06.428265    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod823aa414_5ccc_468c_95d7_6207de93926d.slice/crio-ae872680133dbd85cd5bcebd1ed59a4f95d18edf7718e6ec70a06b9c6c3352c9 WatchSource:0}: Error finding container ae872680133dbd85cd5bcebd1ed59a4f95d18edf7718e6ec70a06b9c6c3352c9: Status 404 returned error can't find the container with id ae872680133dbd85cd5bcebd1ed59a4f95d18edf7718e6ec70a06b9c6c3352c9
Sep 05 06:44:06.498182 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.498149    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:06.512305 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.512282    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"]
Sep 05 06:44:06.517049 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.517031    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:44:06.519833 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.519812    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-39047b-dockercfg-qf9ll\""
Sep 05 06:44:06.519833 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.519828    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-39047b\""
Sep 05 06:44:06.526584 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.526561    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"]
Sep 05 06:44:06.538539 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.538518    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"]
Sep 05 06:44:06.542077 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.542059    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:44:06.546356 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.546334    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2" event={"ID":"823aa414-5ccc-468c-95d7-6207de93926d","Type":"ContainerStarted","Data":"ae872680133dbd85cd5bcebd1ed59a4f95d18edf7718e6ec70a06b9c6c3352c9"}
Sep 05 06:44:06.554741 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.554722    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"]
Sep 05 06:44:06.642213 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.642175    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbr4b\" (UniqueName: \"kubernetes.io/projected/5082c60e-3dc9-497e-a709-07cf4bcfd827-kube-api-access-cbr4b\") pod \"payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4\" (UID: \"5082c60e-3dc9-497e-a709-07cf4bcfd827\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:44:06.642358 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.642254    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5082c60e-3dc9-497e-a709-07cf4bcfd827-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4\" (UID: \"5082c60e-3dc9-497e-a709-07cf4bcfd827\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:44:06.642358 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.642303    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rzcp\" (UniqueName: \"kubernetes.io/projected/1f999151-f340-4c43-a201-536b610b9048-kube-api-access-5rzcp\") pod \"payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl\" (UID: \"1f999151-f340-4c43-a201-536b610b9048\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:44:06.642358 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.642331    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f999151-f340-4c43-a201-536b610b9048-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl\" (UID: \"1f999151-f340-4c43-a201-536b610b9048\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:44:06.743203 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.743164    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbr4b\" (UniqueName: \"kubernetes.io/projected/5082c60e-3dc9-497e-a709-07cf4bcfd827-kube-api-access-cbr4b\") pod \"payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4\" (UID: \"5082c60e-3dc9-497e-a709-07cf4bcfd827\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:44:06.743642 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.743243    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5082c60e-3dc9-497e-a709-07cf4bcfd827-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4\" (UID: \"5082c60e-3dc9-497e-a709-07cf4bcfd827\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:44:06.743642 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.743313    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rzcp\" (UniqueName: \"kubernetes.io/projected/1f999151-f340-4c43-a201-536b610b9048-kube-api-access-5rzcp\") pod \"payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl\" (UID: \"1f999151-f340-4c43-a201-536b610b9048\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:44:06.743642 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.743355    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f999151-f340-4c43-a201-536b610b9048-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl\" (UID: \"1f999151-f340-4c43-a201-536b610b9048\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:44:06.743989 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.743970    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5082c60e-3dc9-497e-a709-07cf4bcfd827-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4\" (UID: \"5082c60e-3dc9-497e-a709-07cf4bcfd827\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:44:06.744301 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.744278    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f999151-f340-4c43-a201-536b610b9048-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl\" (UID: \"1f999151-f340-4c43-a201-536b610b9048\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:44:06.753167 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.753112    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbr4b\" (UniqueName: \"kubernetes.io/projected/5082c60e-3dc9-497e-a709-07cf4bcfd827-kube-api-access-cbr4b\") pod \"payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4\" (UID: \"5082c60e-3dc9-497e-a709-07cf4bcfd827\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:44:06.753167 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.753129    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rzcp\" (UniqueName: \"kubernetes.io/projected/1f999151-f340-4c43-a201-536b610b9048-kube-api-access-5rzcp\") pod \"payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl\" (UID: \"1f999151-f340-4c43-a201-536b610b9048\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:44:06.828746 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.828714    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:44:06.852090 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:06.852064    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:44:07.105545 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.105503    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"]
Sep 05 06:44:07.106250 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:44:07.106216    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5082c60e_3dc9_497e_a709_07cf4bcfd827.slice/crio-7463333cb666ccda6abcf9a106b4e947b0158f7ce1fbfb08ae7106fecccf4bd8 WatchSource:0}: Error finding container 7463333cb666ccda6abcf9a106b4e947b0158f7ce1fbfb08ae7106fecccf4bd8: Status 404 returned error can't find the container with id 7463333cb666ccda6abcf9a106b4e947b0158f7ce1fbfb08ae7106fecccf4bd8
Sep 05 06:44:07.133402 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.133377    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"]
Sep 05 06:44:07.139009 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:44:07.138972    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f999151_f340_4c43_a201_536b610b9048.slice/crio-3a8e678a8489daed30a4182e99746cec101b2a8afe3bd3ab36e2c03895b3087d WatchSource:0}: Error finding container 3a8e678a8489daed30a4182e99746cec101b2a8afe3bd3ab36e2c03895b3087d: Status 404 returned error can't find the container with id 3a8e678a8489daed30a4182e99746cec101b2a8afe3bd3ab36e2c03895b3087d
Sep 05 06:44:07.552749 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.552712    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl" event={"ID":"1f999151-f340-4c43-a201-536b610b9048","Type":"ContainerStarted","Data":"41f87e921cd62b0dcefeea80e1af7814aea908b2ad83755a4085feb34150eb79"}
Sep 05 06:44:07.552990 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.552777    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl" event={"ID":"1f999151-f340-4c43-a201-536b610b9048","Type":"ContainerStarted","Data":"3a8e678a8489daed30a4182e99746cec101b2a8afe3bd3ab36e2c03895b3087d"}
Sep 05 06:44:07.552990 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.552875    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:44:07.554614 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.554588    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2" event={"ID":"823aa414-5ccc-468c-95d7-6207de93926d","Type":"ContainerStarted","Data":"19cec060e539fbf7eb473106ea956133ea87df39631b98d32524e265e576fa83"}
Sep 05 06:44:07.554734 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.554702    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:07.556383 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.556361    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4" event={"ID":"5082c60e-3dc9-497e-a709-07cf4bcfd827","Type":"ContainerStarted","Data":"7ea4958b6379da4bb1085151e5f74f6178489226b95866dd1e6bf06b4763986b"}
Sep 05 06:44:07.556497 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.556404    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4" event={"ID":"5082c60e-3dc9-497e-a709-07cf4bcfd827","Type":"ContainerStarted","Data":"7463333cb666ccda6abcf9a106b4e947b0158f7ce1fbfb08ae7106fecccf4bd8"}
Sep 05 06:44:07.556497 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.556428    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:44:07.571801 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.571754    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl" podStartSLOduration=1.571741287 podStartE2EDuration="1.571741287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:44:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:44:07.569520496 +0000 UTC m=+1095.937567708" watchObservedRunningTime="2026-09-05 06:44:07.571741287 +0000 UTC m=+1095.939788499"
Sep 05 06:44:07.588344 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.588301    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2" podStartSLOduration=2.017699059 podStartE2EDuration="2.588283664s" totalImagesPullingTime="570.584605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:44:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:44:07.586776454 +0000 UTC m=+1095.954823663" watchObservedRunningTime="2026-09-05 06:44:07.588283664 +0000 UTC m=+1095.956330874"
Sep 05 06:44:07.608524 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:07.608466    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4" podStartSLOduration=1.60844971 podStartE2EDuration="1.60844971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:44:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:44:07.607816374 +0000 UTC m=+1095.975863586" watchObservedRunningTime="2026-09-05 06:44:07.60844971 +0000 UTC m=+1095.976496919"
Sep 05 06:44:12.244712 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:12.244681    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:12.314462 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:12.314431    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:12.417199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:12.417162    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:12.513113 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:12.513027    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:12.628184 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:12.628150    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:13.566084 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:13.566054    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:44:18.562834 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:18.562804    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:44:18.563282 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:18.562917    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:44:18.941902 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:18.941806    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:19.069382 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:19.069345    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:19.895173 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:19.895143    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:20.207871 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:20.207759    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:20.696677 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:20.696643    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:23.812440 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:23.812408    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:23.911667 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:23.911633    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:24.933843 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:24.933814    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"]
Sep 05 06:44:24.937394 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:24.937372    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:24.940053 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:24.940031    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-362c80-serving-cert\""
Sep 05 06:44:24.949727 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:24.949705    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"]
Sep 05 06:44:24.975157 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:24.975132    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:25.013372 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.013347    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/085e1960-0483-488e-8c5f-04517cdc0def-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.013504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.013380    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6sbk\" (UniqueName: \"kubernetes.io/projected/085e1960-0483-488e-8c5f-04517cdc0def-kube-api-access-m6sbk\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.013504 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.013410    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.014077 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.014045    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.082313 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.082274    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:25.115279 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.115253    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/085e1960-0483-488e-8c5f-04517cdc0def-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.115403 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.115286    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6sbk\" (UniqueName: \"kubernetes.io/projected/085e1960-0483-488e-8c5f-04517cdc0def-kube-api-access-m6sbk\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.115403 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.115307    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.115403 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.115373    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.115566 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:44:25.115496    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-serving-cert: secret "maas-api-e2e-shared-a-w2-362c80-serving-cert" not found
Sep 05 06:44:25.115566 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:44:25.115560    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-maas-api-tls podName:085e1960-0483-488e-8c5f-04517cdc0def nodeName:}" failed. No retries permitted until 2026-09-05 06:44:25.615541772 +0000 UTC m=+1113.983588967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-maas-api-tls") pod "maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh" (UID: "085e1960-0483-488e-8c5f-04517cdc0def") : secret "maas-api-e2e-shared-a-w2-362c80-serving-cert" not found
Sep 05 06:44:25.115955 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.115927    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/085e1960-0483-488e-8c5f-04517cdc0def-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.117788 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.117763    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.128959 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.128936    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6sbk\" (UniqueName: \"kubernetes.io/projected/085e1960-0483-488e-8c5f-04517cdc0def-kube-api-access-m6sbk\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.184389 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.184326    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:25.506207 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.506174    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:25.608688 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.608656    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:25.619672 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.619643    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.621997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.621976    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.641777 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.641747    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"]
Sep 05 06:44:25.645920 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.645901    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:44:25.648311 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.648282    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-362c80\""
Sep 05 06:44:25.648427 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.648375    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-362c80-dockercfg-gndcq\""
Sep 05 06:44:25.655827 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.655802    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"]
Sep 05 06:44:25.820885 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.820766    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce4adc1-09c9-4235-8518-3a91615cfd0e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g\" (UID: \"9ce4adc1-09c9-4235-8518-3a91615cfd0e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:44:25.820885 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.820819    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpc6b\" (UniqueName: \"kubernetes.io/projected/9ce4adc1-09c9-4235-8518-3a91615cfd0e-kube-api-access-vpc6b\") pod \"payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g\" (UID: \"9ce4adc1-09c9-4235-8518-3a91615cfd0e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:44:25.850087 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.850052    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:25.922020 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.921770    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce4adc1-09c9-4235-8518-3a91615cfd0e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g\" (UID: \"9ce4adc1-09c9-4235-8518-3a91615cfd0e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:44:25.922020 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.921817    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpc6b\" (UniqueName: \"kubernetes.io/projected/9ce4adc1-09c9-4235-8518-3a91615cfd0e-kube-api-access-vpc6b\") pod \"payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g\" (UID: \"9ce4adc1-09c9-4235-8518-3a91615cfd0e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:44:25.934339 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.923011    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce4adc1-09c9-4235-8518-3a91615cfd0e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g\" (UID: \"9ce4adc1-09c9-4235-8518-3a91615cfd0e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:44:25.935517 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.935459    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpc6b\" (UniqueName: \"kubernetes.io/projected/9ce4adc1-09c9-4235-8518-3a91615cfd0e-kube-api-access-vpc6b\") pod \"payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g\" (UID: \"9ce4adc1-09c9-4235-8518-3a91615cfd0e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:44:25.958065 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:25.958032    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:44:26.127771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:26.127734    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"]
Sep 05 06:44:26.129714 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:44:26.129684    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod085e1960_0483_488e_8c5f_04517cdc0def.slice/crio-ce8bff9bfa6eea5d967ded13a5e410c71c0611a2213fa5c3ddc3b84fb2eab3cb WatchSource:0}: Error finding container ce8bff9bfa6eea5d967ded13a5e410c71c0611a2213fa5c3ddc3b84fb2eab3cb: Status 404 returned error can't find the container with id ce8bff9bfa6eea5d967ded13a5e410c71c0611a2213fa5c3ddc3b84fb2eab3cb
Sep 05 06:44:26.231414 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:26.231391    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"]
Sep 05 06:44:26.233679 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:44:26.233646    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce4adc1_09c9_4235_8518_3a91615cfd0e.slice/crio-79e9cda8d345b1c633644b2bef59ff850f4b1da37fd2e5935faa7de98c8b5974 WatchSource:0}: Error finding container 79e9cda8d345b1c633644b2bef59ff850f4b1da37fd2e5935faa7de98c8b5974: Status 404 returned error can't find the container with id 79e9cda8d345b1c633644b2bef59ff850f4b1da37fd2e5935faa7de98c8b5974
Sep 05 06:44:26.652884 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:26.652830    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh" event={"ID":"085e1960-0483-488e-8c5f-04517cdc0def","Type":"ContainerStarted","Data":"2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976"}
Sep 05 06:44:26.653038 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:26.652920    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:26.653038 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:26.652946    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh" event={"ID":"085e1960-0483-488e-8c5f-04517cdc0def","Type":"ContainerStarted","Data":"ce8bff9bfa6eea5d967ded13a5e410c71c0611a2213fa5c3ddc3b84fb2eab3cb"}
Sep 05 06:44:26.654546 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:26.654521    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g" event={"ID":"9ce4adc1-09c9-4235-8518-3a91615cfd0e","Type":"ContainerStarted","Data":"023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445"}
Sep 05 06:44:26.654651 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:26.654567    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g" event={"ID":"9ce4adc1-09c9-4235-8518-3a91615cfd0e","Type":"ContainerStarted","Data":"79e9cda8d345b1c633644b2bef59ff850f4b1da37fd2e5935faa7de98c8b5974"}
Sep 05 06:44:26.654651 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:26.654587    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:44:26.672630 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:26.672590    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh" podStartSLOduration=2.261565307 podStartE2EDuration="2.672576056s" totalImagesPullingTime="411.010749ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:44:26.670292111 +0000 UTC m=+1115.038339334" watchObservedRunningTime="2026-09-05 06:44:26.672576056 +0000 UTC m=+1115.040623265"
Sep 05 06:44:26.688214 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:26.688167    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g" podStartSLOduration=1.688148821 podStartE2EDuration="1.688148821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:44:26.687107813 +0000 UTC m=+1115.055155024" watchObservedRunningTime="2026-09-05 06:44:26.688148821 +0000 UTC m=+1115.056196034"
Sep 05 06:44:27.045772 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:27.045740    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:29.553550 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:29.553510    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:32.666736 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:32.666701    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:44:37.663460 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:37.663429    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:44:40.628942 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:40.628907    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:46.041083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:46.041047    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:48.714203 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:48.714164    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:49.054113 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:49.054075    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:49.401910 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:49.401797    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:49.505266 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:49.505225    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:52.276756 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:52.276727    2573 scope.go:117] "RemoveContainer" containerID="df56e34a9a11eb8dc8fe022a7daf8da5820a88469244976659de6b4afe3d79c1"
Sep 05 06:44:52.944537 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:52.944502    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:53.033938 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:53.033894    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:54.207009 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.206974    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"]
Sep 05 06:44:54.212728 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.212703    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.215256 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.215231    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-5e47e8-serving-cert\""
Sep 05 06:44:54.222484 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.222459    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"]
Sep 05 06:44:54.266256 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.266224    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:54.282691 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.282653    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.282909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.282706    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.282909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.282815    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khfn8\" (UniqueName: \"kubernetes.io/projected/d9f7ffeb-0c76-44db-b704-c47aab7c581c-kube-api-access-khfn8\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.283039 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.282918    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f7ffeb-0c76-44db-b704-c47aab7c581c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.378600 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.378561    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:54.383828 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.383803    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.383943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.383876    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.384000 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:44:54.383940    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-serving-cert: secret "maas-api-e2e-shared-b-w2-5e47e8-serving-cert" not found
Sep 05 06:44:54.384000 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:44:54.383991    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-maas-api-tls podName:d9f7ffeb-0c76-44db-b704-c47aab7c581c nodeName:}" failed. No retries permitted until 2026-09-05 06:44:54.883977817 +0000 UTC m=+1143.252025006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-maas-api-tls") pod "maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr" (UID: "d9f7ffeb-0c76-44db-b704-c47aab7c581c") : secret "maas-api-e2e-shared-b-w2-5e47e8-serving-cert" not found
Sep 05 06:44:54.384097 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.383941    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khfn8\" (UniqueName: \"kubernetes.io/projected/d9f7ffeb-0c76-44db-b704-c47aab7c581c-kube-api-access-khfn8\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.384143 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.384099    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f7ffeb-0c76-44db-b704-c47aab7c581c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.384699 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.384676    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f7ffeb-0c76-44db-b704-c47aab7c581c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.386470 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.386447    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.395483 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.395464    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khfn8\" (UniqueName: \"kubernetes.io/projected/d9f7ffeb-0c76-44db-b704-c47aab7c581c-kube-api-access-khfn8\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.633233 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.633186    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:54.890234 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.890147    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:54.892480 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:54.892463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:55.046457 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:55.046411    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:55.127526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:55.127485    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:55.159585 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:55.159496    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:44:55.416560 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:55.416531    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"]
Sep 05 06:44:55.418614 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:44:55.418582    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f7ffeb_0c76_44db_b704_c47aab7c581c.slice/crio-adf5a04946d69783c3d291a5e5486dfbd5f475a2384ed94c5cd42874a2f9bf1b WatchSource:0}: Error finding container adf5a04946d69783c3d291a5e5486dfbd5f475a2384ed94c5cd42874a2f9bf1b: Status 404 returned error can't find the container with id adf5a04946d69783c3d291a5e5486dfbd5f475a2384ed94c5cd42874a2f9bf1b
Sep 05 06:44:55.800268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:55.800234    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr" event={"ID":"d9f7ffeb-0c76-44db-b704-c47aab7c581c","Type":"ContainerStarted","Data":"adf5a04946d69783c3d291a5e5486dfbd5f475a2384ed94c5cd42874a2f9bf1b"}
Sep 05 06:44:56.807083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:56.807051    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr" event={"ID":"d9f7ffeb-0c76-44db-b704-c47aab7c581c","Type":"ContainerStarted","Data":"4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70"}
Sep 05 06:44:56.807549 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:56.807176    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:44:56.831072 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:44:56.831022    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr" podStartSLOduration=2.485355294 podStartE2EDuration="2.831006011s" totalImagesPullingTime="345.650717ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:44:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:44:56.828442846 +0000 UTC m=+1145.196490055" watchObservedRunningTime="2026-09-05 06:44:56.831006011 +0000 UTC m=+1145.199053221"
Sep 05 06:45:02.771289 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:02.771249    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:02.817751 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:02.817721    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:45:17.997552 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:17.997466    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:33.106117 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:33.106077    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:33.425827 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:33.425726    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:33.741258 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:33.741228    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:34.576267 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:34.576234    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:36.283439 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:36.283399    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:36.382391 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:36.382361    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:37.487986 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.487951    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"]
Sep 05 06:45:37.492597 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.492574    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.495059 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.495035    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-87058ae4-serving-cert\""
Sep 05 06:45:37.504203 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.504180    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"]
Sep 05 06:45:37.518268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.518245    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:37.584348 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.584314    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdqf9\" (UniqueName: \"kubernetes.io/projected/3cc550de-2d6b-47b6-a601-065f9751521f-kube-api-access-vdqf9\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.584543 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.584384    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc550de-2d6b-47b6-a601-065f9751521f-openshift-service-ca\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.584543 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.584427    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-maas-api-tls\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.584543 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.584489    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-metrics-certs\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.617912 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.617877    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:37.685640 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.685608    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdqf9\" (UniqueName: \"kubernetes.io/projected/3cc550de-2d6b-47b6-a601-065f9751521f-kube-api-access-vdqf9\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.685827 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.685668    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc550de-2d6b-47b6-a601-065f9751521f-openshift-service-ca\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.685827 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.685711    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-maas-api-tls\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.685827 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.685745    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-metrics-certs\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.686320 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.686295    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc550de-2d6b-47b6-a601-065f9751521f-openshift-service-ca\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.688166 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.688142    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-metrics-certs\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.688268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.688184    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-maas-api-tls\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.694192 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.694165    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdqf9\" (UniqueName: \"kubernetes.io/projected/3cc550de-2d6b-47b6-a601-065f9751521f-kube-api-access-vdqf9\") pod \"maas-api-e2e-mt-87058ae4-574776746c-vfm5w\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:37.717582 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.717553    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:37.806972 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:37.806935    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:38.039494 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.039460    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:38.082344 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.082278    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"]
Sep 05 06:45:38.084645 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:45:38.084615    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc550de_2d6b_47b6_a601_065f9751521f.slice/crio-b20307e12cd7a2a928108c99dd946dd9e9413652597457beeda2ae2ccc14e8c1 WatchSource:0}: Error finding container b20307e12cd7a2a928108c99dd946dd9e9413652597457beeda2ae2ccc14e8c1: Status 404 returned error can't find the container with id b20307e12cd7a2a928108c99dd946dd9e9413652597457beeda2ae2ccc14e8c1
Sep 05 06:45:38.137655 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.137625    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:38.228720 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.228690    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"]
Sep 05 06:45:38.233616 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.233594    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:45:38.236038 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.236017    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-87058ae4-dockercfg-4qw8r\""
Sep 05 06:45:38.236141 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.236118    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-87058ae4\""
Sep 05 06:45:38.242486 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.242455    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"]
Sep 05 06:45:38.292667 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.292628    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzdcj\" (UniqueName: \"kubernetes.io/projected/44046dfb-d31e-4eb6-bd88-f89f463d7c72-kube-api-access-qzdcj\") pod \"payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26\" (UID: \"44046dfb-d31e-4eb6-bd88-f89f463d7c72\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:45:38.292878 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.292860    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44046dfb-d31e-4eb6-bd88-f89f463d7c72-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26\" (UID: \"44046dfb-d31e-4eb6-bd88-f89f463d7c72\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:45:38.393936 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.393839    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44046dfb-d31e-4eb6-bd88-f89f463d7c72-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26\" (UID: \"44046dfb-d31e-4eb6-bd88-f89f463d7c72\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:45:38.394102 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.393955    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzdcj\" (UniqueName: \"kubernetes.io/projected/44046dfb-d31e-4eb6-bd88-f89f463d7c72-kube-api-access-qzdcj\") pod \"payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26\" (UID: \"44046dfb-d31e-4eb6-bd88-f89f463d7c72\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:45:38.394589 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.394569    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44046dfb-d31e-4eb6-bd88-f89f463d7c72-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26\" (UID: \"44046dfb-d31e-4eb6-bd88-f89f463d7c72\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:45:38.403166 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.403145    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzdcj\" (UniqueName: \"kubernetes.io/projected/44046dfb-d31e-4eb6-bd88-f89f463d7c72-kube-api-access-qzdcj\") pod \"payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26\" (UID: \"44046dfb-d31e-4eb6-bd88-f89f463d7c72\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:45:38.546241 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.546210    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:45:38.827061 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:38.827018    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"]
Sep 05 06:45:38.830662 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:45:38.830633    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44046dfb_d31e_4eb6_bd88_f89f463d7c72.slice/crio-1e71bc632630eaf9d6ab56de3d7f0a9d24ac600109c21a090cb85d93f1f2047c WatchSource:0}: Error finding container 1e71bc632630eaf9d6ab56de3d7f0a9d24ac600109c21a090cb85d93f1f2047c: Status 404 returned error can't find the container with id 1e71bc632630eaf9d6ab56de3d7f0a9d24ac600109c21a090cb85d93f1f2047c
Sep 05 06:45:39.020945 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:39.020908    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w" event={"ID":"3cc550de-2d6b-47b6-a601-065f9751521f","Type":"ContainerStarted","Data":"4560847e12ae42f381b3b5337caa7a13d90d72a026ebbb4d1584a5fffc762327"}
Sep 05 06:45:39.021119 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:39.020974    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:39.021119 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:39.020995    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w" event={"ID":"3cc550de-2d6b-47b6-a601-065f9751521f","Type":"ContainerStarted","Data":"b20307e12cd7a2a928108c99dd946dd9e9413652597457beeda2ae2ccc14e8c1"}
Sep 05 06:45:39.022377 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:39.022354    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26" event={"ID":"44046dfb-d31e-4eb6-bd88-f89f463d7c72","Type":"ContainerStarted","Data":"70193e849a4ef36b7439b3e962359f122c04ef1bf6de592ce0a463df763461d9"}
Sep 05 06:45:39.022464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:39.022397    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26" event={"ID":"44046dfb-d31e-4eb6-bd88-f89f463d7c72","Type":"ContainerStarted","Data":"1e71bc632630eaf9d6ab56de3d7f0a9d24ac600109c21a090cb85d93f1f2047c"}
Sep 05 06:45:39.022464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:39.022417    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:45:39.040258 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:39.040210    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w" podStartSLOduration=1.5552321199999999 podStartE2EDuration="2.040197696s" totalImagesPullingTime="484.965576ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:45:39.038800448 +0000 UTC m=+1187.406847657" watchObservedRunningTime="2026-09-05 06:45:39.040197696 +0000 UTC m=+1187.408244904"
Sep 05 06:45:39.056514 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:39.056463    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26" podStartSLOduration=1.056450406 podStartE2EDuration="1.056450406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:45:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:45:39.055694755 +0000 UTC m=+1187.423741989" watchObservedRunningTime="2026-09-05 06:45:39.056450406 +0000 UTC m=+1187.424497641"
Sep 05 06:45:43.815716 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:43.815684    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:45.033692 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:45.033665    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:45:46.469155 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:46.469120    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:50.029544 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:50.029514    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:45:50.853056 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:50.853020    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:50.953336 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:50.953301    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:45:52.162418 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:52.162393    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:45:52.164968 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:52.164945    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:45:58.588083 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:45:58.588045    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:46:01.339903 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:46:01.339871    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:46:29.717532 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:46:29.717489    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:47:17.221775 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.221679    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"]
Sep 05 06:47:17.222429 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.222119    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w" podUID="3cc550de-2d6b-47b6-a601-065f9751521f" containerName="maas-api" containerID="cri-o://4560847e12ae42f381b3b5337caa7a13d90d72a026ebbb4d1584a5fffc762327" gracePeriod=30
Sep 05 06:47:17.253700 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.253663    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:47:17.294489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.294452    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"]
Sep 05 06:47:17.294731 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.294706    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26" podUID="44046dfb-d31e-4eb6-bd88-f89f463d7c72" containerName="payload-pre-processing" containerID="cri-o://70193e849a4ef36b7439b3e962359f122c04ef1bf6de592ce0a463df763461d9" gracePeriod=30
Sep 05 06:47:17.356003 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.355964    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:47:17.453681 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.453644    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:47:17.521313 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.521079    2573 generic.go:356] "Generic (PLEG): container finished" podID="44046dfb-d31e-4eb6-bd88-f89f463d7c72" containerID="70193e849a4ef36b7439b3e962359f122c04ef1bf6de592ce0a463df763461d9" exitCode=0
Sep 05 06:47:17.521313 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.521283    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26" event={"ID":"44046dfb-d31e-4eb6-bd88-f89f463d7c72","Type":"ContainerDied","Data":"70193e849a4ef36b7439b3e962359f122c04ef1bf6de592ce0a463df763461d9"}
Sep 05 06:47:17.524599 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.524470    2573 generic.go:356] "Generic (PLEG): container finished" podID="3cc550de-2d6b-47b6-a601-065f9751521f" containerID="4560847e12ae42f381b3b5337caa7a13d90d72a026ebbb4d1584a5fffc762327" exitCode=0
Sep 05 06:47:17.524599 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.524536    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w" event={"ID":"3cc550de-2d6b-47b6-a601-065f9751521f","Type":"ContainerDied","Data":"4560847e12ae42f381b3b5337caa7a13d90d72a026ebbb4d1584a5fffc762327"}
Sep 05 06:47:17.525050 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.525005    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:47:17.561374 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.561345    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:47:17.573499 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.573332    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-maas-api-tls\") pod \"3cc550de-2d6b-47b6-a601-065f9751521f\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") "
Sep 05 06:47:17.573499 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.573365    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-metrics-certs\") pod \"3cc550de-2d6b-47b6-a601-065f9751521f\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") "
Sep 05 06:47:17.573499 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.573409    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdqf9\" (UniqueName: \"kubernetes.io/projected/3cc550de-2d6b-47b6-a601-065f9751521f-kube-api-access-vdqf9\") pod \"3cc550de-2d6b-47b6-a601-065f9751521f\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") "
Sep 05 06:47:17.573499 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.573457    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc550de-2d6b-47b6-a601-065f9751521f-openshift-service-ca\") pod \"3cc550de-2d6b-47b6-a601-065f9751521f\" (UID: \"3cc550de-2d6b-47b6-a601-065f9751521f\") "
Sep 05 06:47:17.574078 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.574043    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cc550de-2d6b-47b6-a601-065f9751521f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3cc550de-2d6b-47b6-a601-065f9751521f" (UID: "3cc550de-2d6b-47b6-a601-065f9751521f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:47:17.575624 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.575595    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cc550de-2d6b-47b6-a601-065f9751521f-kube-api-access-vdqf9" (OuterVolumeSpecName: "kube-api-access-vdqf9") pod "3cc550de-2d6b-47b6-a601-065f9751521f" (UID: "3cc550de-2d6b-47b6-a601-065f9751521f"). InnerVolumeSpecName "kube-api-access-vdqf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:47:17.575719 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.575680    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3cc550de-2d6b-47b6-a601-065f9751521f" (UID: "3cc550de-2d6b-47b6-a601-065f9751521f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:47:17.575772 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.575755    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3cc550de-2d6b-47b6-a601-065f9751521f" (UID: "3cc550de-2d6b-47b6-a601-065f9751521f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:47:17.593045 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.593022    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:47:17.674634 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.674603    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44046dfb-d31e-4eb6-bd88-f89f463d7c72-plugins-config-volume\") pod \"44046dfb-d31e-4eb6-bd88-f89f463d7c72\" (UID: \"44046dfb-d31e-4eb6-bd88-f89f463d7c72\") "
Sep 05 06:47:17.674828 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.674675    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzdcj\" (UniqueName: \"kubernetes.io/projected/44046dfb-d31e-4eb6-bd88-f89f463d7c72-kube-api-access-qzdcj\") pod \"44046dfb-d31e-4eb6-bd88-f89f463d7c72\" (UID: \"44046dfb-d31e-4eb6-bd88-f89f463d7c72\") "
Sep 05 06:47:17.675064 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.675045    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:47:17.675125 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.675060    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44046dfb-d31e-4eb6-bd88-f89f463d7c72-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "44046dfb-d31e-4eb6-bd88-f89f463d7c72" (UID: "44046dfb-d31e-4eb6-bd88-f89f463d7c72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:47:17.675125 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.675069    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc550de-2d6b-47b6-a601-065f9751521f-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:47:17.675125 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.675111    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdqf9\" (UniqueName: \"kubernetes.io/projected/3cc550de-2d6b-47b6-a601-065f9751521f-kube-api-access-vdqf9\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:47:17.675272 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.675124    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc550de-2d6b-47b6-a601-065f9751521f-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:47:17.676819 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.676795    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44046dfb-d31e-4eb6-bd88-f89f463d7c72-kube-api-access-qzdcj" (OuterVolumeSpecName: "kube-api-access-qzdcj") pod "44046dfb-d31e-4eb6-bd88-f89f463d7c72" (UID: "44046dfb-d31e-4eb6-bd88-f89f463d7c72"). InnerVolumeSpecName "kube-api-access-qzdcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:47:17.776098 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.776011    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44046dfb-d31e-4eb6-bd88-f89f463d7c72-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:47:17.776098 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:17.776040    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzdcj\" (UniqueName: \"kubernetes.io/projected/44046dfb-d31e-4eb6-bd88-f89f463d7c72-kube-api-access-qzdcj\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:47:18.534423 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:18.534391    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"
Sep 05 06:47:18.534946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:18.534420    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26" event={"ID":"44046dfb-d31e-4eb6-bd88-f89f463d7c72","Type":"ContainerDied","Data":"1e71bc632630eaf9d6ab56de3d7f0a9d24ac600109c21a090cb85d93f1f2047c"}
Sep 05 06:47:18.534946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:18.534496    2573 scope.go:117] "RemoveContainer" containerID="70193e849a4ef36b7439b3e962359f122c04ef1bf6de592ce0a463df763461d9"
Sep 05 06:47:18.535997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:18.535808    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w" event={"ID":"3cc550de-2d6b-47b6-a601-065f9751521f","Type":"ContainerDied","Data":"b20307e12cd7a2a928108c99dd946dd9e9413652597457beeda2ae2ccc14e8c1"}
Sep 05 06:47:18.535997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:18.535829    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"
Sep 05 06:47:18.543464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:18.543447    2573 scope.go:117] "RemoveContainer" containerID="4560847e12ae42f381b3b5337caa7a13d90d72a026ebbb4d1584a5fffc762327"
Sep 05 06:47:18.552766 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:18.552741    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"]
Sep 05 06:47:18.557061 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:18.557041    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87058ae4-574776746c-vfm5w"]
Sep 05 06:47:18.568569 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:18.568547    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"]
Sep 05 06:47:18.572906 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:18.572887    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87058ae4-66587b5c98-g2k26"]
Sep 05 06:47:20.155665 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:20.155634    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cc550de-2d6b-47b6-a601-065f9751521f" path="/var/lib/kubelet/pods/3cc550de-2d6b-47b6-a601-065f9751521f/volumes"
Sep 05 06:47:20.156050 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:20.156006    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44046dfb-d31e-4eb6-bd88-f89f463d7c72" path="/var/lib/kubelet/pods/44046dfb-d31e-4eb6-bd88-f89f463d7c72/volumes"
Sep 05 06:47:21.125703 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:21.125666    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:47:26.198184 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:26.198145    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:47:40.591456 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.591423    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7741-84e036"]
Sep 05 06:47:40.591845 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.591833    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44046dfb-d31e-4eb6-bd88-f89f463d7c72" containerName="payload-pre-processing"
Sep 05 06:47:40.591845 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.591846    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="44046dfb-d31e-4eb6-bd88-f89f463d7c72" containerName="payload-pre-processing"
Sep 05 06:47:40.591845 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.591870    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cc550de-2d6b-47b6-a601-065f9751521f" containerName="maas-api"
Sep 05 06:47:40.591845 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.591876    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cc550de-2d6b-47b6-a601-065f9751521f" containerName="maas-api"
Sep 05 06:47:40.592039 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.591961    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="44046dfb-d31e-4eb6-bd88-f89f463d7c72" containerName="payload-pre-processing"
Sep 05 06:47:40.592039 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.591972    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cc550de-2d6b-47b6-a601-065f9751521f" containerName="maas-api"
Sep 05 06:47:40.595234 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.595217    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-84e036"
Sep 05 06:47:40.597781 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.597758    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wwpm9\""
Sep 05 06:47:40.602954 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.602644    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7741-84e036"]
Sep 05 06:47:40.699007 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.698956    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2w5q\" (UniqueName: \"kubernetes.io/projected/e2c515d3-ae52-412a-8b1c-706283959468-kube-api-access-z2w5q\") pod \"mt-curl-7741-84e036\" (UID: \"e2c515d3-ae52-412a-8b1c-706283959468\") " pod="opendatahub/mt-curl-7741-84e036"
Sep 05 06:47:40.800041 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.800004    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2w5q\" (UniqueName: \"kubernetes.io/projected/e2c515d3-ae52-412a-8b1c-706283959468-kube-api-access-z2w5q\") pod \"mt-curl-7741-84e036\" (UID: \"e2c515d3-ae52-412a-8b1c-706283959468\") " pod="opendatahub/mt-curl-7741-84e036"
Sep 05 06:47:40.809383 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.809360    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2w5q\" (UniqueName: \"kubernetes.io/projected/e2c515d3-ae52-412a-8b1c-706283959468-kube-api-access-z2w5q\") pod \"mt-curl-7741-84e036\" (UID: \"e2c515d3-ae52-412a-8b1c-706283959468\") " pod="opendatahub/mt-curl-7741-84e036"
Sep 05 06:47:40.906415 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:40.906340    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-84e036"
Sep 05 06:47:41.182125 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:41.182105    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7741-84e036"]
Sep 05 06:47:41.184302 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:47:41.184276    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2c515d3_ae52_412a_8b1c_706283959468.slice/crio-baf783600430d3fac4a476787b128570f1b1c19673872a74bd7a9d965ad98b74 WatchSource:0}: Error finding container baf783600430d3fac4a476787b128570f1b1c19673872a74bd7a9d965ad98b74: Status 404 returned error can't find the container with id baf783600430d3fac4a476787b128570f1b1c19673872a74bd7a9d965ad98b74
Sep 05 06:47:41.652836 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:41.652799    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7741-84e036" event={"ID":"e2c515d3-ae52-412a-8b1c-706283959468","Type":"ContainerStarted","Data":"baf783600430d3fac4a476787b128570f1b1c19673872a74bd7a9d965ad98b74"}
Sep 05 06:47:42.659952 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:42.659918    2573 generic.go:356] "Generic (PLEG): container finished" podID="e2c515d3-ae52-412a-8b1c-706283959468" containerID="1fd89ddfee3534a369c690ef4104bd3e8c8da318e9c16716a30c976f24a697b9" exitCode=0
Sep 05 06:47:42.660422 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:42.659988    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7741-84e036" event={"ID":"e2c515d3-ae52-412a-8b1c-706283959468","Type":"ContainerDied","Data":"1fd89ddfee3534a369c690ef4104bd3e8c8da318e9c16716a30c976f24a697b9"}
Sep 05 06:47:43.797487 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:43.797460    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-84e036"
Sep 05 06:47:43.822213 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:43.822161    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7741-84e036_e2c515d3-ae52-412a-8b1c-706283959468/mt-curl-7741-84e036/0.log"
Sep 05 06:47:43.847832 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:43.847801    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7741-84e036"]
Sep 05 06:47:43.849646 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:43.849620    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7741-84e036"]
Sep 05 06:47:43.928262 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:43.928165    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2w5q\" (UniqueName: \"kubernetes.io/projected/e2c515d3-ae52-412a-8b1c-706283959468-kube-api-access-z2w5q\") pod \"e2c515d3-ae52-412a-8b1c-706283959468\" (UID: \"e2c515d3-ae52-412a-8b1c-706283959468\") "
Sep 05 06:47:43.930272 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:43.930249    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2c515d3-ae52-412a-8b1c-706283959468-kube-api-access-z2w5q" (OuterVolumeSpecName: "kube-api-access-z2w5q") pod "e2c515d3-ae52-412a-8b1c-706283959468" (UID: "e2c515d3-ae52-412a-8b1c-706283959468"). InnerVolumeSpecName "kube-api-access-z2w5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:47:44.029214 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:44.029175    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2w5q\" (UniqueName: \"kubernetes.io/projected/e2c515d3-ae52-412a-8b1c-706283959468-kube-api-access-z2w5q\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:47:44.155933 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:44.155901    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2c515d3-ae52-412a-8b1c-706283959468" path="/var/lib/kubelet/pods/e2c515d3-ae52-412a-8b1c-706283959468/volumes"
Sep 05 06:47:44.671501 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:44.671475    2573 scope.go:117] "RemoveContainer" containerID="1fd89ddfee3534a369c690ef4104bd3e8c8da318e9c16716a30c976f24a697b9"
Sep 05 06:47:44.671691 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:44.671476    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-84e036"
Sep 05 06:47:45.337557 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:45.337520    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:47:46.489830 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:47:46.489790    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:02.224395 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:02.224320    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:03.332871 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:03.332823    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:09.282318 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:09.282279    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:18.244371 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:18.244277    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:24.097475 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:24.097438    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:24.194330 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:24.194296    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:24.357577 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:24.357465    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:25.639432 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:25.639398    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:25.745578 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:25.745529    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:25.854900 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:25.854864    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:26.446522 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:26.446484    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:26.545925 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:26.545890    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:26.645981 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:26.645947    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:27.936731 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:27.936692    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:34.233340 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:34.233303    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:34.299514 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:34.299481    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:34.940208 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:34.940170    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:38.715988 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:38.715952    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:41.538420 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:41.538365    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:41.600170 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:41.600133    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:41.668045 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:41.668000    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:41.766918 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:41.766874    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:52.228396 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:52.228354    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:52.468294 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:52.468261    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:48:59.353514 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:48:59.353481    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:13.422494 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:13.422440    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:19.776735 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:19.776694    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:19.875273 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:19.875237    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:20.140694 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:20.140582    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:20.465806 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:20.465709    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:20.870159 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:20.870123    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:23.624292 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:23.624245    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:23.705706 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:23.705665    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:24.758385 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.758351    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"]
Sep 05 06:49:24.758948 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.758904    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2c515d3-ae52-412a-8b1c-706283959468" containerName="mt-curl-7741-84e036"
Sep 05 06:49:24.758948 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.758924    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2c515d3-ae52-412a-8b1c-706283959468" containerName="mt-curl-7741-84e036"
Sep 05 06:49:24.759059 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.759046    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2c515d3-ae52-412a-8b1c-706283959468" containerName="mt-curl-7741-84e036"
Sep 05 06:49:24.762318 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.762298    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:24.765066 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.765044    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-96d352-serving-cert\""
Sep 05 06:49:24.775322 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.775296    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"]
Sep 05 06:49:24.791201 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.791160    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:24.797303 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.797247    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-metrics-certs\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:24.797303 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.797295    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ffa04c6-13d2-426e-9aa0-842995d1158b-openshift-service-ca\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:24.797567 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.797351    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfmgr\" (UniqueName: \"kubernetes.io/projected/9ffa04c6-13d2-426e-9aa0-842995d1158b-kube-api-access-zfmgr\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:24.797567 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.797485    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-maas-api-tls\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:24.891569 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.891528    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:24.898751 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.898719    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-metrics-certs\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:24.898913 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.898766    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ffa04c6-13d2-426e-9aa0-842995d1158b-openshift-service-ca\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:24.898913 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.898811    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfmgr\" (UniqueName: \"kubernetes.io/projected/9ffa04c6-13d2-426e-9aa0-842995d1158b-kube-api-access-zfmgr\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:24.899024 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.898923    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-maas-api-tls\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:24.899083 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:49:24.899046    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-serving-cert: secret "maas-api-e2e-api-a-96d352-serving-cert" not found
Sep 05 06:49:24.899136 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:49:24.899120    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-maas-api-tls podName:9ffa04c6-13d2-426e-9aa0-842995d1158b nodeName:}" failed. No retries permitted until 2026-09-05 06:49:25.399099209 +0000 UTC m=+1413.767146400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-maas-api-tls") pod "maas-api-e2e-api-a-96d352-8498b8d965-d6srr" (UID: "9ffa04c6-13d2-426e-9aa0-842995d1158b") : secret "maas-api-e2e-api-a-96d352-serving-cert" not found
Sep 05 06:49:24.899555 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.899533    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ffa04c6-13d2-426e-9aa0-842995d1158b-openshift-service-ca\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:24.901196 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.901173    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-metrics-certs\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:24.910361 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:24.910333    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfmgr\" (UniqueName: \"kubernetes.io/projected/9ffa04c6-13d2-426e-9aa0-842995d1158b-kube-api-access-zfmgr\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:25.186787 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.186704    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:25.338666 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.338630    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:25.405118 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.405085    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-maas-api-tls\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:25.407543 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.407519    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-maas-api-tls\") pod \"maas-api-e2e-api-a-96d352-8498b8d965-d6srr\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:25.434263 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.434236    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:25.519557 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.519515    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"]
Sep 05 06:49:25.523358 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.523336    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:49:25.525691 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.525668    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-96d352\""
Sep 05 06:49:25.525824 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.525722    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-96d352-dockercfg-xb85d\""
Sep 05 06:49:25.534576 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.534549    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"]
Sep 05 06:49:25.607040 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.606996    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvfmd\" (UniqueName: \"kubernetes.io/projected/3d2c7927-f7ad-459d-bd83-49ba27c6db58-kube-api-access-rvfmd\") pod \"payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59\" (UID: \"3d2c7927-f7ad-459d-bd83-49ba27c6db58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:49:25.607040 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.607042    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d2c7927-f7ad-459d-bd83-49ba27c6db58-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59\" (UID: \"3d2c7927-f7ad-459d-bd83-49ba27c6db58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:49:25.675651 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.675617    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:25.707752 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.707714    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvfmd\" (UniqueName: \"kubernetes.io/projected/3d2c7927-f7ad-459d-bd83-49ba27c6db58-kube-api-access-rvfmd\") pod \"payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59\" (UID: \"3d2c7927-f7ad-459d-bd83-49ba27c6db58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:49:25.707927 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.707772    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d2c7927-f7ad-459d-bd83-49ba27c6db58-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59\" (UID: \"3d2c7927-f7ad-459d-bd83-49ba27c6db58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:49:25.708540 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.708521    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d2c7927-f7ad-459d-bd83-49ba27c6db58-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59\" (UID: \"3d2c7927-f7ad-459d-bd83-49ba27c6db58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:49:25.718444 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.718384    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvfmd\" (UniqueName: \"kubernetes.io/projected/3d2c7927-f7ad-459d-bd83-49ba27c6db58-kube-api-access-rvfmd\") pod \"payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59\" (UID: \"3d2c7927-f7ad-459d-bd83-49ba27c6db58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:49:25.834034 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.833946    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:49:25.957357 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.957331    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"]
Sep 05 06:49:25.960267 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:49:25.960233    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ffa04c6_13d2_426e_9aa0_842995d1158b.slice/crio-b6d0b5aee39c2a3c4622f5aef1c43b368a7321234e93165c261a761d05376452 WatchSource:0}: Error finding container b6d0b5aee39c2a3c4622f5aef1c43b368a7321234e93165c261a761d05376452: Status 404 returned error can't find the container with id b6d0b5aee39c2a3c4622f5aef1c43b368a7321234e93165c261a761d05376452
Sep 05 06:49:25.962119 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:25.962100    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 06:49:26.125362 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:26.125334    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"]
Sep 05 06:49:26.127324 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:49:26.127295    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d2c7927_f7ad_459d_bd83_49ba27c6db58.slice/crio-34779289696a3cfcacb6a5e48d6b6429e3c3ab8f6ed86904bdd9b0e5d940d0bb WatchSource:0}: Error finding container 34779289696a3cfcacb6a5e48d6b6429e3c3ab8f6ed86904bdd9b0e5d940d0bb: Status 404 returned error can't find the container with id 34779289696a3cfcacb6a5e48d6b6429e3c3ab8f6ed86904bdd9b0e5d940d0bb
Sep 05 06:49:26.175647 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:26.175608    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59" event={"ID":"3d2c7927-f7ad-459d-bd83-49ba27c6db58","Type":"ContainerStarted","Data":"34779289696a3cfcacb6a5e48d6b6429e3c3ab8f6ed86904bdd9b0e5d940d0bb"}
Sep 05 06:49:26.177032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:26.176970    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr" event={"ID":"9ffa04c6-13d2-426e-9aa0-842995d1158b","Type":"ContainerStarted","Data":"b6d0b5aee39c2a3c4622f5aef1c43b368a7321234e93165c261a761d05376452"}
Sep 05 06:49:26.548383 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:26.548349    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:27.184669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:27.184638    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59" event={"ID":"3d2c7927-f7ad-459d-bd83-49ba27c6db58","Type":"ContainerStarted","Data":"a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1"}
Sep 05 06:49:27.185177 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:27.184771    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:49:27.186454 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:27.186430    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr" event={"ID":"9ffa04c6-13d2-426e-9aa0-842995d1158b","Type":"ContainerStarted","Data":"ae7fad7219075755d80527629a609075b5bb8235e66187b9ab8be5513b1aa71b"}
Sep 05 06:49:27.186564 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:27.186486    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:27.206046 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:27.205999    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59" podStartSLOduration=2.205984026 podStartE2EDuration="2.205984026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:49:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:49:27.204036684 +0000 UTC m=+1415.572083895" watchObservedRunningTime="2026-09-05 06:49:27.205984026 +0000 UTC m=+1415.574031263"
Sep 05 06:49:27.222826 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:27.222782    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr" podStartSLOduration=2.797021142 podStartE2EDuration="3.222767197s" totalImagesPullingTime="425.746055ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:49:27.221356038 +0000 UTC m=+1415.589403250" watchObservedRunningTime="2026-09-05 06:49:27.222767197 +0000 UTC m=+1415.590814407"
Sep 05 06:49:27.842131 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:27.842097    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:33.197800 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:33.197765    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:49:38.194283 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:38.194254    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:49:41.188976 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:41.188930    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:45.869441 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:45.869360    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:46.304069 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:46.304037    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:47.333020 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:47.332984    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:47.615531 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:47.615429    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:48.058104 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:48.058075    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:48.563388 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:48.563352    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:49.016000 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:49.015959    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:49.329097 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:49.328988    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:49.394799 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:49.394766    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:52.789846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:52.789807    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:52.896916 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:52.896872    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:53.912938 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:53.912906    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"]
Sep 05 06:49:53.916754 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:53.916733    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:53.919045 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:53.919016    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a9301f-serving-cert\""
Sep 05 06:49:53.927167 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:53.927139    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"]
Sep 05 06:49:53.953324 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:53.953288    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:53.994049 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:53.994022    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-maas-api-tls\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:53.994200 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:53.994077    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-metrics-certs\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:53.994200 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:53.994119    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n46n6\" (UniqueName: \"kubernetes.io/projected/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-kube-api-access-n46n6\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:53.994200 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:53.994143    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-openshift-service-ca\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:54.053250 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.053210    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:54.094971 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.094932    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-maas-api-tls\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:54.095190 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.094990    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-metrics-certs\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:54.095190 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.095023    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n46n6\" (UniqueName: \"kubernetes.io/projected/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-kube-api-access-n46n6\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:54.095190 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.095047    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-openshift-service-ca\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:54.095797 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.095775    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-openshift-service-ca\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:54.097542 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.097518    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-metrics-certs\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:54.097542 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.097537    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-maas-api-tls\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:54.104824 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.104797    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n46n6\" (UniqueName: \"kubernetes.io/projected/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-kube-api-access-n46n6\") pod \"maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:54.152799 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.152764    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:54.231089 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.231001    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:54.449101 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.449047    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:54.503862 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.503826    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"]
Sep 05 06:49:54.505640 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:49:54.505607    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc14db62c_9380_41fe_8ce3_b20e40cb7f6f.slice/crio-dcd41bf1a2a825285ec904fbaaa3759cade43e0c4ca8f32464a73820fa6a92e4 WatchSource:0}: Error finding container dcd41bf1a2a825285ec904fbaaa3759cade43e0c4ca8f32464a73820fa6a92e4: Status 404 returned error can't find the container with id dcd41bf1a2a825285ec904fbaaa3759cade43e0c4ca8f32464a73820fa6a92e4
Sep 05 06:49:54.551339 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:54.551305    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:49:55.334479 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:55.334437    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct" event={"ID":"c14db62c-9380-41fe-8ce3-b20e40cb7f6f","Type":"ContainerStarted","Data":"7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125"}
Sep 05 06:49:55.334887 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:55.334508    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct" event={"ID":"c14db62c-9380-41fe-8ce3-b20e40cb7f6f","Type":"ContainerStarted","Data":"dcd41bf1a2a825285ec904fbaaa3759cade43e0c4ca8f32464a73820fa6a92e4"}
Sep 05 06:49:55.334887 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:55.334552    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:49:55.353339 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:49:55.353288    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct" podStartSLOduration=1.972993462 podStartE2EDuration="2.35327205s" totalImagesPullingTime="380.278588ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:49:55.351585586 +0000 UTC m=+1443.719632797" watchObservedRunningTime="2026-09-05 06:49:55.35327205 +0000 UTC m=+1443.721319259"
Sep 05 06:50:01.346242 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:01.346216    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:50:03.508233 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:03.508205    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:17.635205 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:17.635164    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:17.763440 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:17.763406    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:18.156918 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:18.156882    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:20.837380 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:20.837344    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:36.393018 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.392988    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"]
Sep 05 06:50:36.393587 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.393195    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr" podUID="9ffa04c6-13d2-426e-9aa0-842995d1158b" containerName="maas-api" containerID="cri-o://ae7fad7219075755d80527629a609075b5bb8235e66187b9ab8be5513b1aa71b" gracePeriod=30
Sep 05 06:50:36.423503 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.423468    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:36.464864 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.464815    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"]
Sep 05 06:50:36.465135 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.465108    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59" podUID="3d2c7927-f7ad-459d-bd83-49ba27c6db58" containerName="payload-pre-processing" containerID="cri-o://a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1" gracePeriod=30
Sep 05 06:50:36.523347 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.523314    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:36.546802 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.546773    2573 generic.go:356] "Generic (PLEG): container finished" podID="9ffa04c6-13d2-426e-9aa0-842995d1158b" containerID="ae7fad7219075755d80527629a609075b5bb8235e66187b9ab8be5513b1aa71b" exitCode=0
Sep 05 06:50:36.546963 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.546847    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr" event={"ID":"9ffa04c6-13d2-426e-9aa0-842995d1158b","Type":"ContainerDied","Data":"ae7fad7219075755d80527629a609075b5bb8235e66187b9ab8be5513b1aa71b"}
Sep 05 06:50:36.623726 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.623653    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:36.682215 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.681399    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:36.716704 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.716684    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:50:36.743776 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.743752    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:50:36.816922 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.816897    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-maas-api-tls\") pod \"9ffa04c6-13d2-426e-9aa0-842995d1158b\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") "
Sep 05 06:50:36.817106 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.816934    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvfmd\" (UniqueName: \"kubernetes.io/projected/3d2c7927-f7ad-459d-bd83-49ba27c6db58-kube-api-access-rvfmd\") pod \"3d2c7927-f7ad-459d-bd83-49ba27c6db58\" (UID: \"3d2c7927-f7ad-459d-bd83-49ba27c6db58\") "
Sep 05 06:50:36.817106 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.816961    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d2c7927-f7ad-459d-bd83-49ba27c6db58-plugins-config-volume\") pod \"3d2c7927-f7ad-459d-bd83-49ba27c6db58\" (UID: \"3d2c7927-f7ad-459d-bd83-49ba27c6db58\") "
Sep 05 06:50:36.817106 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.817013    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ffa04c6-13d2-426e-9aa0-842995d1158b-openshift-service-ca\") pod \"9ffa04c6-13d2-426e-9aa0-842995d1158b\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") "
Sep 05 06:50:36.817106 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.817055    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfmgr\" (UniqueName: \"kubernetes.io/projected/9ffa04c6-13d2-426e-9aa0-842995d1158b-kube-api-access-zfmgr\") pod \"9ffa04c6-13d2-426e-9aa0-842995d1158b\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") "
Sep 05 06:50:36.817106 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.817092    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-metrics-certs\") pod \"9ffa04c6-13d2-426e-9aa0-842995d1158b\" (UID: \"9ffa04c6-13d2-426e-9aa0-842995d1158b\") "
Sep 05 06:50:36.817452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.817410    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ffa04c6-13d2-426e-9aa0-842995d1158b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ffa04c6-13d2-426e-9aa0-842995d1158b" (UID: "9ffa04c6-13d2-426e-9aa0-842995d1158b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:50:36.817452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.817417    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d2c7927-f7ad-459d-bd83-49ba27c6db58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3d2c7927-f7ad-459d-bd83-49ba27c6db58" (UID: "3d2c7927-f7ad-459d-bd83-49ba27c6db58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:50:36.819081 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.819051    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ffa04c6-13d2-426e-9aa0-842995d1158b" (UID: "9ffa04c6-13d2-426e-9aa0-842995d1158b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:50:36.819196 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.819119    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d2c7927-f7ad-459d-bd83-49ba27c6db58-kube-api-access-rvfmd" (OuterVolumeSpecName: "kube-api-access-rvfmd") pod "3d2c7927-f7ad-459d-bd83-49ba27c6db58" (UID: "3d2c7927-f7ad-459d-bd83-49ba27c6db58"). InnerVolumeSpecName "kube-api-access-rvfmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:50:36.819196 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.819126    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9ffa04c6-13d2-426e-9aa0-842995d1158b" (UID: "9ffa04c6-13d2-426e-9aa0-842995d1158b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:50:36.819196 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.819139    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ffa04c6-13d2-426e-9aa0-842995d1158b-kube-api-access-zfmgr" (OuterVolumeSpecName: "kube-api-access-zfmgr") pod "9ffa04c6-13d2-426e-9aa0-842995d1158b" (UID: "9ffa04c6-13d2-426e-9aa0-842995d1158b"). InnerVolumeSpecName "kube-api-access-zfmgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:50:36.918489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.918463    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfmgr\" (UniqueName: \"kubernetes.io/projected/9ffa04c6-13d2-426e-9aa0-842995d1158b-kube-api-access-zfmgr\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:50:36.918489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.918486    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:50:36.918489 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.918495    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ffa04c6-13d2-426e-9aa0-842995d1158b-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:50:36.918690 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.918504    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvfmd\" (UniqueName: \"kubernetes.io/projected/3d2c7927-f7ad-459d-bd83-49ba27c6db58-kube-api-access-rvfmd\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:50:36.918690 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.918514    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d2c7927-f7ad-459d-bd83-49ba27c6db58-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:50:36.918690 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.918524    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ffa04c6-13d2-426e-9aa0-842995d1158b-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:50:36.987243 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:36.987142    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:37.554173 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.554137    2573 generic.go:356] "Generic (PLEG): container finished" podID="3d2c7927-f7ad-459d-bd83-49ba27c6db58" containerID="a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1" exitCode=0
Sep 05 06:50:37.554619 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.554188    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"
Sep 05 06:50:37.554619 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.554226    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59" event={"ID":"3d2c7927-f7ad-459d-bd83-49ba27c6db58","Type":"ContainerDied","Data":"a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1"}
Sep 05 06:50:37.554619 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.554279    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59" event={"ID":"3d2c7927-f7ad-459d-bd83-49ba27c6db58","Type":"ContainerDied","Data":"34779289696a3cfcacb6a5e48d6b6429e3c3ab8f6ed86904bdd9b0e5d940d0bb"}
Sep 05 06:50:37.554619 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.554307    2573 scope.go:117] "RemoveContainer" containerID="a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1"
Sep 05 06:50:37.555800 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.555783    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"
Sep 05 06:50:37.555946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.555875    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr" event={"ID":"9ffa04c6-13d2-426e-9aa0-842995d1158b","Type":"ContainerDied","Data":"b6d0b5aee39c2a3c4622f5aef1c43b368a7321234e93165c261a761d05376452"}
Sep 05 06:50:37.563695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.563675    2573 scope.go:117] "RemoveContainer" containerID="a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1"
Sep 05 06:50:37.564020 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:50:37.563995    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1\": container with ID starting with a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1 not found: ID does not exist" containerID="a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1"
Sep 05 06:50:37.564094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.564033    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1"} err="failed to get container status \"a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1\": rpc error: code = NotFound desc = could not find container \"a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1\": container with ID starting with a01bb0df8db1b8da39dc01421973dbcc3be42c37bda2df83a6702ee1549e7fa1 not found: ID does not exist"
Sep 05 06:50:37.564094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.564057    2573 scope.go:117] "RemoveContainer" containerID="ae7fad7219075755d80527629a609075b5bb8235e66187b9ab8be5513b1aa71b"
Sep 05 06:50:37.579555 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.579534    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"]
Sep 05 06:50:37.584616 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.584589    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-96d352-66b7c448f8-8kh59"]
Sep 05 06:50:37.596275 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.596252    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"]
Sep 05 06:50:37.598917 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:37.598900    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-96d352-8498b8d965-d6srr"]
Sep 05 06:50:38.155743 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:38.155703    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d2c7927-f7ad-459d-bd83-49ba27c6db58" path="/var/lib/kubelet/pods/3d2c7927-f7ad-459d-bd83-49ba27c6db58/volumes"
Sep 05 06:50:38.156203 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:38.156183    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ffa04c6-13d2-426e-9aa0-842995d1158b" path="/var/lib/kubelet/pods/9ffa04c6-13d2-426e-9aa0-842995d1158b/volumes"
Sep 05 06:50:42.412704 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:42.412660    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:43.644474 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:43.644437    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:43.742494 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:43.742463    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:43.961683 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:43.961594    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:44.390510 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:44.390480    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:47.237614 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:47.237568    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:50:52.193301 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:52.193274    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:50:52.197520 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:52.197493    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:50:54.536010 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:50:54.535936    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:00.136803 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.136762    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"]
Sep 05 06:51:00.137326 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.137039    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct" podUID="c14db62c-9380-41fe-8ce3-b20e40cb7f6f" containerName="maas-api" containerID="cri-o://7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125" gracePeriod=30
Sep 05 06:51:00.168482 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.168443    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:00.252659 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.252624    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:00.313133 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.313099    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:00.390463 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.390395    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:51:00.390888 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.390846    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:00.453258 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.453220    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:00.537746 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.537719    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-openshift-service-ca\") pod \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") "
Sep 05 06:51:00.537935 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.537755    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n46n6\" (UniqueName: \"kubernetes.io/projected/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-kube-api-access-n46n6\") pod \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") "
Sep 05 06:51:00.537935 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.537818    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-maas-api-tls\") pod \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") "
Sep 05 06:51:00.537935 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.537910    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-metrics-certs\") pod \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\" (UID: \"c14db62c-9380-41fe-8ce3-b20e40cb7f6f\") "
Sep 05 06:51:00.538116 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.538093    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c14db62c-9380-41fe-8ce3-b20e40cb7f6f" (UID: "c14db62c-9380-41fe-8ce3-b20e40cb7f6f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:51:00.538241 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.538222    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:00.540030 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.540004    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-kube-api-access-n46n6" (OuterVolumeSpecName: "kube-api-access-n46n6") pod "c14db62c-9380-41fe-8ce3-b20e40cb7f6f" (UID: "c14db62c-9380-41fe-8ce3-b20e40cb7f6f"). InnerVolumeSpecName "kube-api-access-n46n6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:51:00.540140 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.540009    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c14db62c-9380-41fe-8ce3-b20e40cb7f6f" (UID: "c14db62c-9380-41fe-8ce3-b20e40cb7f6f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:51:00.540140 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.540029    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c14db62c-9380-41fe-8ce3-b20e40cb7f6f" (UID: "c14db62c-9380-41fe-8ce3-b20e40cb7f6f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:51:00.639481 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.639438    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:00.639481 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.639475    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n46n6\" (UniqueName: \"kubernetes.io/projected/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-kube-api-access-n46n6\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:00.639481 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.639484    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c14db62c-9380-41fe-8ce3-b20e40cb7f6f-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:00.675119 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.675047    2573 generic.go:356] "Generic (PLEG): container finished" podID="c14db62c-9380-41fe-8ce3-b20e40cb7f6f" containerID="7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125" exitCode=0
Sep 05 06:51:00.675119 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.675101    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"
Sep 05 06:51:00.675279 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.675128    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct" event={"ID":"c14db62c-9380-41fe-8ce3-b20e40cb7f6f","Type":"ContainerDied","Data":"7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125"}
Sep 05 06:51:00.675279 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.675195    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct" event={"ID":"c14db62c-9380-41fe-8ce3-b20e40cb7f6f","Type":"ContainerDied","Data":"dcd41bf1a2a825285ec904fbaaa3759cade43e0c4ca8f32464a73820fa6a92e4"}
Sep 05 06:51:00.675279 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.675215    2573 scope.go:117] "RemoveContainer" containerID="7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125"
Sep 05 06:51:00.684771 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.684753    2573 scope.go:117] "RemoveContainer" containerID="7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125"
Sep 05 06:51:00.685047 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:51:00.685029    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125\": container with ID starting with 7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125 not found: ID does not exist" containerID="7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125"
Sep 05 06:51:00.685113 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.685054    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125"} err="failed to get container status \"7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125\": rpc error: code = NotFound desc = could not find container \"7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125\": container with ID starting with 7738203a567f41bea03fcbee4d0dd8b7cfab8ee1bf3193338578765ba7ff6125 not found: ID does not exist"
Sep 05 06:51:00.697524 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.697501    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"]
Sep 05 06:51:00.702297 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.702277    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a9301f-66566df8fc-gt4ct"]
Sep 05 06:51:00.750423 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:00.750383    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:02.156639 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:02.156610    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c14db62c-9380-41fe-8ce3-b20e40cb7f6f" path="/var/lib/kubelet/pods/c14db62c-9380-41fe-8ce3-b20e40cb7f6f/volumes"
Sep 05 06:51:06.830625 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:06.830589    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:06.891636 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:06.891598    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:07.084475 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:07.084380    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:07.517009 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:07.516974    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:17.905934 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:17.905860    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"]
Sep 05 06:51:17.906350 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:17.906089    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh" podUID="085e1960-0483-488e-8c5f-04517cdc0def" containerName="maas-api" containerID="cri-o://2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976" gracePeriod=30
Sep 05 06:51:17.937232 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:17.937201    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:17.968356 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:17.968324    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"]
Sep 05 06:51:17.968595 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:17.968570    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g" podUID="9ce4adc1-09c9-4235-8518-3a91615cfd0e" containerName="payload-processing" containerID="cri-o://023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445" gracePeriod=30
Sep 05 06:51:18.013564 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.013529    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:18.110223 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.110154    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:18.179467 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.179424    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:18.255143 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.255115    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:51:18.258972 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.258953    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:51:18.316077 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.316045    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/085e1960-0483-488e-8c5f-04517cdc0def-openshift-service-ca\") pod \"085e1960-0483-488e-8c5f-04517cdc0def\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") "
Sep 05 06:51:18.316268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.316107    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6sbk\" (UniqueName: \"kubernetes.io/projected/085e1960-0483-488e-8c5f-04517cdc0def-kube-api-access-m6sbk\") pod \"085e1960-0483-488e-8c5f-04517cdc0def\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") "
Sep 05 06:51:18.316268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.316156    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce4adc1-09c9-4235-8518-3a91615cfd0e-plugins-config-volume\") pod \"9ce4adc1-09c9-4235-8518-3a91615cfd0e\" (UID: \"9ce4adc1-09c9-4235-8518-3a91615cfd0e\") "
Sep 05 06:51:18.316268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.316237    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-metrics-certs\") pod \"085e1960-0483-488e-8c5f-04517cdc0def\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") "
Sep 05 06:51:18.316434 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.316297    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-maas-api-tls\") pod \"085e1960-0483-488e-8c5f-04517cdc0def\" (UID: \"085e1960-0483-488e-8c5f-04517cdc0def\") "
Sep 05 06:51:18.316434 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.316322    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpc6b\" (UniqueName: \"kubernetes.io/projected/9ce4adc1-09c9-4235-8518-3a91615cfd0e-kube-api-access-vpc6b\") pod \"9ce4adc1-09c9-4235-8518-3a91615cfd0e\" (UID: \"9ce4adc1-09c9-4235-8518-3a91615cfd0e\") "
Sep 05 06:51:18.316532 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.316472    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/085e1960-0483-488e-8c5f-04517cdc0def-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "085e1960-0483-488e-8c5f-04517cdc0def" (UID: "085e1960-0483-488e-8c5f-04517cdc0def"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:51:18.316609 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.316578    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ce4adc1-09c9-4235-8518-3a91615cfd0e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ce4adc1-09c9-4235-8518-3a91615cfd0e" (UID: "9ce4adc1-09c9-4235-8518-3a91615cfd0e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:51:18.316672 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.316648    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/085e1960-0483-488e-8c5f-04517cdc0def-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:18.318650 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.318625    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "085e1960-0483-488e-8c5f-04517cdc0def" (UID: "085e1960-0483-488e-8c5f-04517cdc0def"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:51:18.318807 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.318787    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/085e1960-0483-488e-8c5f-04517cdc0def-kube-api-access-m6sbk" (OuterVolumeSpecName: "kube-api-access-m6sbk") pod "085e1960-0483-488e-8c5f-04517cdc0def" (UID: "085e1960-0483-488e-8c5f-04517cdc0def"). InnerVolumeSpecName "kube-api-access-m6sbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:51:18.318990 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.318970    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "085e1960-0483-488e-8c5f-04517cdc0def" (UID: "085e1960-0483-488e-8c5f-04517cdc0def"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:51:18.319059 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.318988    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ce4adc1-09c9-4235-8518-3a91615cfd0e-kube-api-access-vpc6b" (OuterVolumeSpecName: "kube-api-access-vpc6b") pod "9ce4adc1-09c9-4235-8518-3a91615cfd0e" (UID: "9ce4adc1-09c9-4235-8518-3a91615cfd0e"). InnerVolumeSpecName "kube-api-access-vpc6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:51:18.417635 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.417600    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:18.417635 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.417633    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpc6b\" (UniqueName: \"kubernetes.io/projected/9ce4adc1-09c9-4235-8518-3a91615cfd0e-kube-api-access-vpc6b\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:18.417635 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.417644    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6sbk\" (UniqueName: \"kubernetes.io/projected/085e1960-0483-488e-8c5f-04517cdc0def-kube-api-access-m6sbk\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:18.417941 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.417653    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce4adc1-09c9-4235-8518-3a91615cfd0e-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:18.417941 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.417661    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085e1960-0483-488e-8c5f-04517cdc0def-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:18.515766 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.515720    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:18.766223 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.766136    2573 generic.go:356] "Generic (PLEG): container finished" podID="085e1960-0483-488e-8c5f-04517cdc0def" containerID="2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976" exitCode=0
Sep 05 06:51:18.766223 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.766205    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"
Sep 05 06:51:18.766451 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.766216    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh" event={"ID":"085e1960-0483-488e-8c5f-04517cdc0def","Type":"ContainerDied","Data":"2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976"}
Sep 05 06:51:18.766451 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.766291    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh" event={"ID":"085e1960-0483-488e-8c5f-04517cdc0def","Type":"ContainerDied","Data":"ce8bff9bfa6eea5d967ded13a5e410c71c0611a2213fa5c3ddc3b84fb2eab3cb"}
Sep 05 06:51:18.766451 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.766316    2573 scope.go:117] "RemoveContainer" containerID="2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976"
Sep 05 06:51:18.767558 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.767541    2573 generic.go:356] "Generic (PLEG): container finished" podID="9ce4adc1-09c9-4235-8518-3a91615cfd0e" containerID="023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445" exitCode=0
Sep 05 06:51:18.767667 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.767572    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g" event={"ID":"9ce4adc1-09c9-4235-8518-3a91615cfd0e","Type":"ContainerDied","Data":"023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445"}
Sep 05 06:51:18.767667 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.767610    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"
Sep 05 06:51:18.767770 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.767611    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g" event={"ID":"9ce4adc1-09c9-4235-8518-3a91615cfd0e","Type":"ContainerDied","Data":"79e9cda8d345b1c633644b2bef59ff850f4b1da37fd2e5935faa7de98c8b5974"}
Sep 05 06:51:18.776686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.776643    2573 scope.go:117] "RemoveContainer" containerID="2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976"
Sep 05 06:51:18.776947 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:51:18.776921    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976\": container with ID starting with 2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976 not found: ID does not exist" containerID="2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976"
Sep 05 06:51:18.777032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.776956    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976"} err="failed to get container status \"2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976\": rpc error: code = NotFound desc = could not find container \"2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976\": container with ID starting with 2f35c8207419892c86581ddaafb7ff4c818f250c2d2ae56524b30509e0567976 not found: ID does not exist"
Sep 05 06:51:18.777032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.776973    2573 scope.go:117] "RemoveContainer" containerID="023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445"
Sep 05 06:51:18.784656 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.784640    2573 scope.go:117] "RemoveContainer" containerID="023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445"
Sep 05 06:51:18.784905 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:51:18.784882    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445\": container with ID starting with 023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445 not found: ID does not exist" containerID="023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445"
Sep 05 06:51:18.784969 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.784909    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445"} err="failed to get container status \"023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445\": rpc error: code = NotFound desc = could not find container \"023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445\": container with ID starting with 023ac31319d0206c0b5de1572c2645265e9fd2c86587bca0b154cdd8d5169445 not found: ID does not exist"
Sep 05 06:51:18.813161 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.813139    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"]
Sep 05 06:51:18.823618 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.823600    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-362c80-8676f49dbf-p4vbh"]
Sep 05 06:51:18.843528 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.843503    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"]
Sep 05 06:51:18.851950 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:18.851932    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-362c80-868d9b8ff7-lps4g"]
Sep 05 06:51:20.156191 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:20.156151    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="085e1960-0483-488e-8c5f-04517cdc0def" path="/var/lib/kubelet/pods/085e1960-0483-488e-8c5f-04517cdc0def/volumes"
Sep 05 06:51:20.156700 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:20.156679    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ce4adc1-09c9-4235-8518-3a91615cfd0e" path="/var/lib/kubelet/pods/9ce4adc1-09c9-4235-8518-3a91615cfd0e/volumes"
Sep 05 06:51:24.779403 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:24.779370    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:24.852519 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:24.852486    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:25.043915 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:25.043819    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:25.644833 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:25.644799    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:32.424156 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:32.424123    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:33.509340 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:33.509304    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:33.893951 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:33.893841    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:34.189450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:34.189349    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:34.612056 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:34.612020    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:36.040210 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.040179    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"]
Sep 05 06:51:36.040750 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.040370    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr" podUID="d9f7ffeb-0c76-44db-b704-c47aab7c581c" containerName="maas-api" containerID="cri-o://4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70" gracePeriod=30
Sep 05 06:51:36.075168 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.075140    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:36.143723 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.143689    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:36.203161 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.203131    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:36.290997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.290942    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:51:36.305261 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.305226    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:36.382799 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.382770    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-metrics-certs\") pod \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") "
Sep 05 06:51:36.382997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.382816    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-maas-api-tls\") pod \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") "
Sep 05 06:51:36.382997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.382883    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khfn8\" (UniqueName: \"kubernetes.io/projected/d9f7ffeb-0c76-44db-b704-c47aab7c581c-kube-api-access-khfn8\") pod \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") "
Sep 05 06:51:36.382997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.382983    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f7ffeb-0c76-44db-b704-c47aab7c581c-openshift-service-ca\") pod \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\" (UID: \"d9f7ffeb-0c76-44db-b704-c47aab7c581c\") "
Sep 05 06:51:36.383357 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.383327    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9f7ffeb-0c76-44db-b704-c47aab7c581c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9f7ffeb-0c76-44db-b704-c47aab7c581c" (UID: "d9f7ffeb-0c76-44db-b704-c47aab7c581c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:51:36.384988 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.384957    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9f7ffeb-0c76-44db-b704-c47aab7c581c-kube-api-access-khfn8" (OuterVolumeSpecName: "kube-api-access-khfn8") pod "d9f7ffeb-0c76-44db-b704-c47aab7c581c" (UID: "d9f7ffeb-0c76-44db-b704-c47aab7c581c"). InnerVolumeSpecName "kube-api-access-khfn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:51:36.384988 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.384981    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9f7ffeb-0c76-44db-b704-c47aab7c581c" (UID: "d9f7ffeb-0c76-44db-b704-c47aab7c581c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:51:36.385121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.385035    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d9f7ffeb-0c76-44db-b704-c47aab7c581c" (UID: "d9f7ffeb-0c76-44db-b704-c47aab7c581c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:51:36.409955 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.409923    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:36.484179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.484147    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9f7ffeb-0c76-44db-b704-c47aab7c581c-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:36.484179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.484173    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:36.484179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.484182    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9f7ffeb-0c76-44db-b704-c47aab7c581c-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:36.484397 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.484190    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khfn8\" (UniqueName: \"kubernetes.io/projected/d9f7ffeb-0c76-44db-b704-c47aab7c581c-kube-api-access-khfn8\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:51:36.645939 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.645837    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:36.860316 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.860280    2573 generic.go:356] "Generic (PLEG): container finished" podID="d9f7ffeb-0c76-44db-b704-c47aab7c581c" containerID="4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70" exitCode=0
Sep 05 06:51:36.860547 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.860334    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"
Sep 05 06:51:36.860547 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.860366    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr" event={"ID":"d9f7ffeb-0c76-44db-b704-c47aab7c581c","Type":"ContainerDied","Data":"4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70"}
Sep 05 06:51:36.860547 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.860419    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr" event={"ID":"d9f7ffeb-0c76-44db-b704-c47aab7c581c","Type":"ContainerDied","Data":"adf5a04946d69783c3d291a5e5486dfbd5f475a2384ed94c5cd42874a2f9bf1b"}
Sep 05 06:51:36.860547 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.860442    2573 scope.go:117] "RemoveContainer" containerID="4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70"
Sep 05 06:51:36.869566 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.869551    2573 scope.go:117] "RemoveContainer" containerID="4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70"
Sep 05 06:51:36.869826 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:51:36.869805    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70\": container with ID starting with 4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70 not found: ID does not exist" containerID="4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70"
Sep 05 06:51:36.869997 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.869835    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70"} err="failed to get container status \"4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70\": rpc error: code = NotFound desc = could not find container \"4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70\": container with ID starting with 4c066fdaadfc069ad137289eab762a36d366e15cd087241cd7f4ef8d5de53d70 not found: ID does not exist"
Sep 05 06:51:36.892497 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.892472    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"]
Sep 05 06:51:36.906024 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:36.905966    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e47e8-5bbfdbd758-cmxsr"]
Sep 05 06:51:37.285141 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:37.285105    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:37.353004 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:37.352976    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:38.156237 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.156204    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9f7ffeb-0c76-44db-b704-c47aab7c581c" path="/var/lib/kubelet/pods/d9f7ffeb-0c76-44db-b704-c47aab7c581c/volumes"
Sep 05 06:51:38.409319 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409218    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"]
Sep 05 06:51:38.409775 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409730    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce4adc1-09c9-4235-8518-3a91615cfd0e" containerName="payload-processing"
Sep 05 06:51:38.409775 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409750    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce4adc1-09c9-4235-8518-3a91615cfd0e" containerName="payload-processing"
Sep 05 06:51:38.409926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409780    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9f7ffeb-0c76-44db-b704-c47aab7c581c" containerName="maas-api"
Sep 05 06:51:38.409926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409789    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9f7ffeb-0c76-44db-b704-c47aab7c581c" containerName="maas-api"
Sep 05 06:51:38.409926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409799    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d2c7927-f7ad-459d-bd83-49ba27c6db58" containerName="payload-pre-processing"
Sep 05 06:51:38.409926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409810    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d2c7927-f7ad-459d-bd83-49ba27c6db58" containerName="payload-pre-processing"
Sep 05 06:51:38.409926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409821    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c14db62c-9380-41fe-8ce3-b20e40cb7f6f" containerName="maas-api"
Sep 05 06:51:38.409926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409829    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c14db62c-9380-41fe-8ce3-b20e40cb7f6f" containerName="maas-api"
Sep 05 06:51:38.409926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409841    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="085e1960-0483-488e-8c5f-04517cdc0def" containerName="maas-api"
Sep 05 06:51:38.409926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409863    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="085e1960-0483-488e-8c5f-04517cdc0def" containerName="maas-api"
Sep 05 06:51:38.409926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409875    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ffa04c6-13d2-426e-9aa0-842995d1158b" containerName="maas-api"
Sep 05 06:51:38.409926 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409882    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ffa04c6-13d2-426e-9aa0-842995d1158b" containerName="maas-api"
Sep 05 06:51:38.410459 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409977    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="085e1960-0483-488e-8c5f-04517cdc0def" containerName="maas-api"
Sep 05 06:51:38.410459 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.409992    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce4adc1-09c9-4235-8518-3a91615cfd0e" containerName="payload-processing"
Sep 05 06:51:38.410459 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.410001    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9f7ffeb-0c76-44db-b704-c47aab7c581c" containerName="maas-api"
Sep 05 06:51:38.410459 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.410010    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c14db62c-9380-41fe-8ce3-b20e40cb7f6f" containerName="maas-api"
Sep 05 06:51:38.410459 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.410019    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ffa04c6-13d2-426e-9aa0-842995d1158b" containerName="maas-api"
Sep 05 06:51:38.410459 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.410028    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d2c7927-f7ad-459d-bd83-49ba27c6db58" containerName="payload-pre-processing"
Sep 05 06:51:38.414414 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.414392    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.417058 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.417038    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-d36c73-serving-cert\""
Sep 05 06:51:38.425453 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.425434    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"]
Sep 05 06:51:38.450281 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.450252    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:38.550252 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.550209    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:38.606262 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.606233    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2wjp\" (UniqueName: \"kubernetes.io/projected/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-kube-api-access-h2wjp\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.606517 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.606325    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.606517 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.606348    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-metrics-certs\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.606517 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.606365    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.650199 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.650158    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:38.707132 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.707056    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2wjp\" (UniqueName: \"kubernetes.io/projected/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-kube-api-access-h2wjp\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.707280 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.707148    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.707280 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.707166    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-metrics-certs\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.707280 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.707186    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.707871 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.707825    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.709499 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.709478    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-metrics-certs\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.709582 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.709547    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.717179 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.717156    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2wjp\" (UniqueName: \"kubernetes.io/projected/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-kube-api-access-h2wjp\") pod \"maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.725918 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.725900    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:38.997273 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:38.997243    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"]
Sep 05 06:51:38.999878 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:51:38.999832    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48bab18b_fd0f_4cde_9843_7a7a9bfa1449.slice/crio-de972b41b857ce8386d02a5ce39812134dcb650546f743824cae97828b89b8ea WatchSource:0}: Error finding container de972b41b857ce8386d02a5ce39812134dcb650546f743824cae97828b89b8ea: Status 404 returned error can't find the container with id de972b41b857ce8386d02a5ce39812134dcb650546f743824cae97828b89b8ea
Sep 05 06:51:39.044998 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.044971    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:39.145113 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.145080    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:39.168372 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.168340    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"]
Sep 05 06:51:39.173087 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.173060    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:51:39.175201 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.175180    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-d36c73-dockercfg-g8cbv\""
Sep 05 06:51:39.175321 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.175205    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-d36c73\""
Sep 05 06:51:39.182413 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.182338    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"]
Sep 05 06:51:39.312785 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.312696    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq9j8\" (UniqueName: \"kubernetes.io/projected/eb301d82-4ec9-4007-88cc-9290f42d267a-kube-api-access-zq9j8\") pod \"payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g\" (UID: \"eb301d82-4ec9-4007-88cc-9290f42d267a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:51:39.312981 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.312892    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb301d82-4ec9-4007-88cc-9290f42d267a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g\" (UID: \"eb301d82-4ec9-4007-88cc-9290f42d267a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:51:39.413820 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.413795    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq9j8\" (UniqueName: \"kubernetes.io/projected/eb301d82-4ec9-4007-88cc-9290f42d267a-kube-api-access-zq9j8\") pod \"payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g\" (UID: \"eb301d82-4ec9-4007-88cc-9290f42d267a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:51:39.414160 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.413938    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb301d82-4ec9-4007-88cc-9290f42d267a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g\" (UID: \"eb301d82-4ec9-4007-88cc-9290f42d267a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:51:39.414806 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.414784    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb301d82-4ec9-4007-88cc-9290f42d267a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g\" (UID: \"eb301d82-4ec9-4007-88cc-9290f42d267a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:51:39.422663 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.422645    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq9j8\" (UniqueName: \"kubernetes.io/projected/eb301d82-4ec9-4007-88cc-9290f42d267a-kube-api-access-zq9j8\") pod \"payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g\" (UID: \"eb301d82-4ec9-4007-88cc-9290f42d267a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:51:39.486321 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.486294    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:51:39.756845 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.756807    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"]
Sep 05 06:51:39.759129 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:51:39.759099    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb301d82_4ec9_4007_88cc_9290f42d267a.slice/crio-51d4f15d56eb9ba5012ce07d6abab0e830167a837bb07600c1f31b7aa2669c5b WatchSource:0}: Error finding container 51d4f15d56eb9ba5012ce07d6abab0e830167a837bb07600c1f31b7aa2669c5b: Status 404 returned error can't find the container with id 51d4f15d56eb9ba5012ce07d6abab0e830167a837bb07600c1f31b7aa2669c5b
Sep 05 06:51:39.878512 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.878466    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g" event={"ID":"eb301d82-4ec9-4007-88cc-9290f42d267a","Type":"ContainerStarted","Data":"a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9"}
Sep 05 06:51:39.878712 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.878536    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g" event={"ID":"eb301d82-4ec9-4007-88cc-9290f42d267a","Type":"ContainerStarted","Data":"51d4f15d56eb9ba5012ce07d6abab0e830167a837bb07600c1f31b7aa2669c5b"}
Sep 05 06:51:39.878712 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.878585    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:51:39.880447 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.880421    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx" event={"ID":"48bab18b-fd0f-4cde-9843-7a7a9bfa1449","Type":"ContainerStarted","Data":"a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40"}
Sep 05 06:51:39.880565 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.880469    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx" event={"ID":"48bab18b-fd0f-4cde-9843-7a7a9bfa1449","Type":"ContainerStarted","Data":"de972b41b857ce8386d02a5ce39812134dcb650546f743824cae97828b89b8ea"}
Sep 05 06:51:39.880565 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.880496    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:39.896409 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.896369    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g" podStartSLOduration=0.89635396 podStartE2EDuration="896.35396ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:51:39.896085578 +0000 UTC m=+1548.264132810" watchObservedRunningTime="2026-09-05 06:51:39.89635396 +0000 UTC m=+1548.264401215"
Sep 05 06:51:39.913050 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:39.912974    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx" podStartSLOduration=1.528299834 podStartE2EDuration="1.912957176s" totalImagesPullingTime="384.657342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:51:39.912752288 +0000 UTC m=+1548.280799501" watchObservedRunningTime="2026-09-05 06:51:39.912957176 +0000 UTC m=+1548.281004385"
Sep 05 06:51:42.834025 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:42.833992    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:42.887991 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:42.887946    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:43.157662 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:43.157579    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:51:45.891424 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:45.891394    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:51:50.888215 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:51:50.888180    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:52:02.988614 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:02.988575    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:03.057089 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:03.057051    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:03.172176 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:03.172146    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:03.463830 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:03.463799    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:06.698218 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:06.698180    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:06.791260 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:06.791221    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:07.921598 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:07.921567    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"]
Sep 05 06:52:07.925229 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:07.925213    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:07.927511 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:07.927494    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-b49086-serving-cert\""
Sep 05 06:52:07.937521 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:07.937500    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"]
Sep 05 06:52:08.016586 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.016557    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:08.099477 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.099445    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d684c23-d8e8-47e0-9fca-892a5fa981c5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.099658 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.099499    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.099658 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.099591    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vpfz\" (UniqueName: \"kubernetes.io/projected/8d684c23-d8e8-47e0-9fca-892a5fa981c5-kube-api-access-6vpfz\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.099658 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.099636    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-metrics-certs\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.115843 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.115813    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:08.201137 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.201041    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-metrics-certs\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.201329 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.201146    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d684c23-d8e8-47e0-9fca-892a5fa981c5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.201329 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.201194    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.201329 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.201268    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vpfz\" (UniqueName: \"kubernetes.io/projected/8d684c23-d8e8-47e0-9fca-892a5fa981c5-kube-api-access-6vpfz\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.201507 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:52:08.201342    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-serving-cert: secret "maas-api-e2e-ipp-b-b49086-serving-cert" not found
Sep 05 06:52:08.201507 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:52:08.201422    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls podName:8d684c23-d8e8-47e0-9fca-892a5fa981c5 nodeName:}" failed. No retries permitted until 2026-09-05 06:52:08.701401775 +0000 UTC m=+1577.069448968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls") pod "maas-api-e2e-ipp-b-b49086-85546bd879-94wp7" (UID: "8d684c23-d8e8-47e0-9fca-892a5fa981c5") : secret "maas-api-e2e-ipp-b-b49086-serving-cert" not found
Sep 05 06:52:08.201916 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.201898    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d684c23-d8e8-47e0-9fca-892a5fa981c5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.203660 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.203637    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-metrics-certs\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.210618 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.210599    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vpfz\" (UniqueName: \"kubernetes.io/projected/8d684c23-d8e8-47e0-9fca-892a5fa981c5-kube-api-access-6vpfz\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.639546 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.639512    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:08.707273 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.707217    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:08.707488 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:52:08.707382    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-serving-cert: secret "maas-api-e2e-ipp-b-b49086-serving-cert" not found
Sep 05 06:52:08.707488 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:52:08.707457    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls podName:8d684c23-d8e8-47e0-9fca-892a5fa981c5 nodeName:}" failed. No retries permitted until 2026-09-05 06:52:09.707437394 +0000 UTC m=+1578.075484582 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls") pod "maas-api-e2e-ipp-b-b49086-85546bd879-94wp7" (UID: "8d684c23-d8e8-47e0-9fca-892a5fa981c5") : secret "maas-api-e2e-ipp-b-b49086-serving-cert" not found
Sep 05 06:52:08.743449 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.743411    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:08.886917 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:08.886846    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:09.717102 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:09.717064    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:09.719459 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:09.719432    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b49086-85546bd879-94wp7\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:09.735085 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:09.735059    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:10.007125 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:10.007100    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"]
Sep 05 06:52:10.009590 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:52:10.009559    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d684c23_d8e8_47e0_9fca_892a5fa981c5.slice/crio-77060ebed0c506f7724913bdb8a8e527d83798ff83db2e05ab03830f7198c373 WatchSource:0}: Error finding container 77060ebed0c506f7724913bdb8a8e527d83798ff83db2e05ab03830f7198c373: Status 404 returned error can't find the container with id 77060ebed0c506f7724913bdb8a8e527d83798ff83db2e05ab03830f7198c373
Sep 05 06:52:10.038390 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:10.038349    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7" event={"ID":"8d684c23-d8e8-47e0-9fca-892a5fa981c5","Type":"ContainerStarted","Data":"77060ebed0c506f7724913bdb8a8e527d83798ff83db2e05ab03830f7198c373"}
Sep 05 06:52:11.045056 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:11.045018    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7" event={"ID":"8d684c23-d8e8-47e0-9fca-892a5fa981c5","Type":"ContainerStarted","Data":"bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9"}
Sep 05 06:52:11.045534 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:11.045122    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:11.067121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:11.067066    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7" podStartSLOduration=3.712845986 podStartE2EDuration="4.067050131s" totalImagesPullingTime="354.204145ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:52:11.066255219 +0000 UTC m=+1579.434302429" watchObservedRunningTime="2026-09-05 06:52:11.067050131 +0000 UTC m=+1579.435097340"
Sep 05 06:52:17.055715 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:17.055684    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:52:26.952267 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:26.952227    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:31.657116 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:31.657085    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:36.672780 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:36.672744    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sslnv_9a9386d4-aa77-4949-af89-303484782499/payload-processing/0.log"
Sep 05 06:52:38.250192 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:38.250157    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:38.622877 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:38.622770    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:42.967841 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:42.967811    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:52:43.041268 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:52:43.041230    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:11.566051 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:11.565966    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g_eb301d82-4ec9-4007-88cc-9290f42d267a/payload-processing/0.log"
Sep 05 06:53:11.699962 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:11.699934    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sslnv_9a9386d4-aa77-4949-af89-303484782499/payload-processing/0.log"
Sep 05 06:53:12.122898 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:12.122836    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:12.432839 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:12.432741    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:12.748863 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:12.748827    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:15.113351 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:15.113313    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:15.215297 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:15.215257    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:16.416994 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.416961    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"]
Sep 05 06:53:16.421125 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.421092    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.423437 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.423413    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-9db0f4-serving-cert\""
Sep 05 06:53:16.433779 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.433753    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"]
Sep 05 06:53:16.454439 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.454406    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:16.554306 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.554264    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:16.566425 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.566404    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkdhn\" (UniqueName: \"kubernetes.io/projected/6607cfc3-9233-4188-843c-c6b3a1ba01de-kube-api-access-kkdhn\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.566532 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.566437    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.566532 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.566482    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.566603 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.566552    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6607cfc3-9233-4188-843c-c6b3a1ba01de-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.616123 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.616090    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:16.667707 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.667632    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.667707 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.667679    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6607cfc3-9233-4188-843c-c6b3a1ba01de-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.667920 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.667733    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkdhn\" (UniqueName: \"kubernetes.io/projected/6607cfc3-9233-4188-843c-c6b3a1ba01de-kube-api-access-kkdhn\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.667920 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.667752    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.668376 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.668354    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6607cfc3-9233-4188-843c-c6b3a1ba01de-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.670041 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.670015    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.670153 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.670133    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.676272 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.676244    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkdhn\" (UniqueName: \"kubernetes.io/projected/6607cfc3-9233-4188-843c-c6b3a1ba01de-kube-api-access-kkdhn\") pod \"maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.732751 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.732713    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:16.941009 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:16.940936    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:17.006936 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.006909    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"]
Sep 05 06:53:17.008965 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:53:17.008940    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6607cfc3_9233_4188_843c_c6b3a1ba01de.slice/crio-19e1cecf80f2fb8bbf4933957335dd4608ebcbf89a9c2c76bc2c6638c6a5bd8e WatchSource:0}: Error finding container 19e1cecf80f2fb8bbf4933957335dd4608ebcbf89a9c2c76bc2c6638c6a5bd8e: Status 404 returned error can't find the container with id 19e1cecf80f2fb8bbf4933957335dd4608ebcbf89a9c2c76bc2c6638c6a5bd8e
Sep 05 06:53:17.014754 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.014727    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:17.134789 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.134760    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"]
Sep 05 06:53:17.139736 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.139717    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:17.142617 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.142591    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-9db0f4\""
Sep 05 06:53:17.142725 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.142700    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-9db0f4-dockercfg-2wljw\""
Sep 05 06:53:17.149343 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.149323    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"]
Sep 05 06:53:17.274200 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.274169    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/887009be-d058-4906-8359-4d6c8f5127ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp\" (UID: \"887009be-d058-4906-8359-4d6c8f5127ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:17.274411 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.274295    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmqph\" (UniqueName: \"kubernetes.io/projected/887009be-d058-4906-8359-4d6c8f5127ff-kube-api-access-zmqph\") pod \"payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp\" (UID: \"887009be-d058-4906-8359-4d6c8f5127ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:17.375144 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.375110    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/887009be-d058-4906-8359-4d6c8f5127ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp\" (UID: \"887009be-d058-4906-8359-4d6c8f5127ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:17.375323 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.375153    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmqph\" (UniqueName: \"kubernetes.io/projected/887009be-d058-4906-8359-4d6c8f5127ff-kube-api-access-zmqph\") pod \"payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp\" (UID: \"887009be-d058-4906-8359-4d6c8f5127ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:17.375920 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.375897    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/887009be-d058-4906-8359-4d6c8f5127ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp\" (UID: \"887009be-d058-4906-8359-4d6c8f5127ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:17.377599 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.377574    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl" event={"ID":"6607cfc3-9233-4188-843c-c6b3a1ba01de","Type":"ContainerStarted","Data":"19e1cecf80f2fb8bbf4933957335dd4608ebcbf89a9c2c76bc2c6638c6a5bd8e"}
Sep 05 06:53:17.385643 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.385623    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmqph\" (UniqueName: \"kubernetes.io/projected/887009be-d058-4906-8359-4d6c8f5127ff-kube-api-access-zmqph\") pod \"payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp\" (UID: \"887009be-d058-4906-8359-4d6c8f5127ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:17.451192 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.451170    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:17.464542 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:53:17.464511    2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6607cfc3_9233_4188_843c_c6b3a1ba01de.slice/crio-24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9.scope\": RecentStats: unable to find data in memory cache]"
Sep 05 06:53:17.729556 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:17.729530    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"]
Sep 05 06:53:17.731250 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:53:17.731222    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod887009be_d058_4906_8359_4d6c8f5127ff.slice/crio-330edf803215a66e8f56aacba72bb105faf8db5e37fd0d8806ac232be55e8a61 WatchSource:0}: Error finding container 330edf803215a66e8f56aacba72bb105faf8db5e37fd0d8806ac232be55e8a61: Status 404 returned error can't find the container with id 330edf803215a66e8f56aacba72bb105faf8db5e37fd0d8806ac232be55e8a61
Sep 05 06:53:18.384469 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:18.384424    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl" event={"ID":"6607cfc3-9233-4188-843c-c6b3a1ba01de","Type":"ContainerStarted","Data":"24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9"}
Sep 05 06:53:18.384654 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:18.384635    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:18.386069 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:18.386037    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp" event={"ID":"887009be-d058-4906-8359-4d6c8f5127ff","Type":"ContainerStarted","Data":"5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356"}
Sep 05 06:53:18.386192 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:18.386081    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp" event={"ID":"887009be-d058-4906-8359-4d6c8f5127ff","Type":"ContainerStarted","Data":"330edf803215a66e8f56aacba72bb105faf8db5e37fd0d8806ac232be55e8a61"}
Sep 05 06:53:18.386192 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:18.386100    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:18.404615 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:18.404576    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl" podStartSLOduration=1.9926179240000002 podStartE2EDuration="2.404562357s" totalImagesPullingTime="411.944433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:53:18.4031626 +0000 UTC m=+1646.771209833" watchObservedRunningTime="2026-09-05 06:53:18.404562357 +0000 UTC m=+1646.772609568"
Sep 05 06:53:18.419492 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:18.419450    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp" podStartSLOduration=1.419433706 podStartE2EDuration="1.419433706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:53:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:53:18.418507006 +0000 UTC m=+1646.786554218" watchObservedRunningTime="2026-09-05 06:53:18.419433706 +0000 UTC m=+1646.787480917"
Sep 05 06:53:24.396819 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:24.396787    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:29.393017 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:29.392972    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:40.134387 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:40.134348    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:47.086893 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.086839    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"]
Sep 05 06:53:47.087758 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.087098    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl" podUID="6607cfc3-9233-4188-843c-c6b3a1ba01de" containerName="maas-api" containerID="cri-o://24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9" gracePeriod=30
Sep 05 06:53:47.117135 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.117101    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:47.157244 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.157211    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"]
Sep 05 06:53:47.157472 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.157448    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp" podUID="887009be-d058-4906-8359-4d6c8f5127ff" containerName="payload-pre-processing" containerID="cri-o://5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356" gracePeriod=30
Sep 05 06:53:47.217142 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.217106    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:47.317341 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.317306    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:47.418368 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.418343    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:47.420757 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.420730    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:47.452990 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.452965    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:47.454837 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.454809    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-maas-api-tls\") pod \"6607cfc3-9233-4188-843c-c6b3a1ba01de\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") "
Sep 05 06:53:47.454995 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.454884    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6607cfc3-9233-4188-843c-c6b3a1ba01de-openshift-service-ca\") pod \"6607cfc3-9233-4188-843c-c6b3a1ba01de\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") "
Sep 05 06:53:47.454995 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.454930    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkdhn\" (UniqueName: \"kubernetes.io/projected/6607cfc3-9233-4188-843c-c6b3a1ba01de-kube-api-access-kkdhn\") pod \"6607cfc3-9233-4188-843c-c6b3a1ba01de\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") "
Sep 05 06:53:47.455117 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.455091    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-metrics-certs\") pod \"6607cfc3-9233-4188-843c-c6b3a1ba01de\" (UID: \"6607cfc3-9233-4188-843c-c6b3a1ba01de\") "
Sep 05 06:53:47.455409 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.455381    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6607cfc3-9233-4188-843c-c6b3a1ba01de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6607cfc3-9233-4188-843c-c6b3a1ba01de" (UID: "6607cfc3-9233-4188-843c-c6b3a1ba01de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:53:47.455838 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.455816    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6607cfc3-9233-4188-843c-c6b3a1ba01de-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:53:47.457710 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.457371    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6607cfc3-9233-4188-843c-c6b3a1ba01de" (UID: "6607cfc3-9233-4188-843c-c6b3a1ba01de"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:53:47.457710 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.457589    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6607cfc3-9233-4188-843c-c6b3a1ba01de" (UID: "6607cfc3-9233-4188-843c-c6b3a1ba01de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:53:47.457710 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.457647    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6607cfc3-9233-4188-843c-c6b3a1ba01de-kube-api-access-kkdhn" (OuterVolumeSpecName: "kube-api-access-kkdhn") pod "6607cfc3-9233-4188-843c-c6b3a1ba01de" (UID: "6607cfc3-9233-4188-843c-c6b3a1ba01de"). InnerVolumeSpecName "kube-api-access-kkdhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:53:47.534130 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.534093    2573 generic.go:356] "Generic (PLEG): container finished" podID="6607cfc3-9233-4188-843c-c6b3a1ba01de" containerID="24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9" exitCode=0
Sep 05 06:53:47.534317 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.534149    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"
Sep 05 06:53:47.534317 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.534177    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl" event={"ID":"6607cfc3-9233-4188-843c-c6b3a1ba01de","Type":"ContainerDied","Data":"24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9"}
Sep 05 06:53:47.534317 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.534238    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl" event={"ID":"6607cfc3-9233-4188-843c-c6b3a1ba01de","Type":"ContainerDied","Data":"19e1cecf80f2fb8bbf4933957335dd4608ebcbf89a9c2c76bc2c6638c6a5bd8e"}
Sep 05 06:53:47.534317 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.534267    2573 scope.go:117] "RemoveContainer" containerID="24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9"
Sep 05 06:53:47.535613 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.535590    2573 generic.go:356] "Generic (PLEG): container finished" podID="887009be-d058-4906-8359-4d6c8f5127ff" containerID="5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356" exitCode=0
Sep 05 06:53:47.535736 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.535621    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp" event={"ID":"887009be-d058-4906-8359-4d6c8f5127ff","Type":"ContainerDied","Data":"5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356"}
Sep 05 06:53:47.535736 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.535676    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp" event={"ID":"887009be-d058-4906-8359-4d6c8f5127ff","Type":"ContainerDied","Data":"330edf803215a66e8f56aacba72bb105faf8db5e37fd0d8806ac232be55e8a61"}
Sep 05 06:53:47.535736 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.535701    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"
Sep 05 06:53:47.543675 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.543659    2573 scope.go:117] "RemoveContainer" containerID="24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9"
Sep 05 06:53:47.544049 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:53:47.544024    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9\": container with ID starting with 24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9 not found: ID does not exist" containerID="24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9"
Sep 05 06:53:47.544166 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.544063    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9"} err="failed to get container status \"24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9\": rpc error: code = NotFound desc = could not find container \"24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9\": container with ID starting with 24256e4f0d6dadd7755ed25612c197755da4aa586785a16c72092de710b8b2c9 not found: ID does not exist"
Sep 05 06:53:47.544166 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.544081    2573 scope.go:117] "RemoveContainer" containerID="5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356"
Sep 05 06:53:47.552230 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.552211    2573 scope.go:117] "RemoveContainer" containerID="5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356"
Sep 05 06:53:47.552464 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:53:47.552442    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356\": container with ID starting with 5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356 not found: ID does not exist" containerID="5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356"
Sep 05 06:53:47.552529 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.552474    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356"} err="failed to get container status \"5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356\": rpc error: code = NotFound desc = could not find container \"5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356\": container with ID starting with 5c5f6f23d2354f99ba883c4b6f5a4a2ea5fa6e790e55887e0bdd7ecc80e66356 not found: ID does not exist"
Sep 05 06:53:47.556846 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.556828    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmqph\" (UniqueName: \"kubernetes.io/projected/887009be-d058-4906-8359-4d6c8f5127ff-kube-api-access-zmqph\") pod \"887009be-d058-4906-8359-4d6c8f5127ff\" (UID: \"887009be-d058-4906-8359-4d6c8f5127ff\") "
Sep 05 06:53:47.556948 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.556937    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/887009be-d058-4906-8359-4d6c8f5127ff-plugins-config-volume\") pod \"887009be-d058-4906-8359-4d6c8f5127ff\" (UID: \"887009be-d058-4906-8359-4d6c8f5127ff\") "
Sep 05 06:53:47.557132 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.557121    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:53:47.557171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.557133    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6607cfc3-9233-4188-843c-c6b3a1ba01de-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:53:47.557171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.557142    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkdhn\" (UniqueName: \"kubernetes.io/projected/6607cfc3-9233-4188-843c-c6b3a1ba01de-kube-api-access-kkdhn\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:53:47.557253 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.557235    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/887009be-d058-4906-8359-4d6c8f5127ff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "887009be-d058-4906-8359-4d6c8f5127ff" (UID: "887009be-d058-4906-8359-4d6c8f5127ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:53:47.558757 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.558737    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/887009be-d058-4906-8359-4d6c8f5127ff-kube-api-access-zmqph" (OuterVolumeSpecName: "kube-api-access-zmqph") pod "887009be-d058-4906-8359-4d6c8f5127ff" (UID: "887009be-d058-4906-8359-4d6c8f5127ff"). InnerVolumeSpecName "kube-api-access-zmqph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:53:47.561959 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.561937    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"]
Sep 05 06:53:47.567121 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.567102    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9db0f4-55fc8bfd57-9svbl"]
Sep 05 06:53:47.658695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.658610    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/887009be-d058-4906-8359-4d6c8f5127ff-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:53:47.658695 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.658639    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmqph\" (UniqueName: \"kubernetes.io/projected/887009be-d058-4906-8359-4d6c8f5127ff-kube-api-access-zmqph\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:53:47.705603 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.705562    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:47.858439 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.858405    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"]
Sep 05 06:53:47.862439 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:47.862412    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9db0f4-6ff5d74794-vdnmp"]
Sep 05 06:53:48.156513 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:48.156477    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6607cfc3-9233-4188-843c-c6b3a1ba01de" path="/var/lib/kubelet/pods/6607cfc3-9233-4188-843c-c6b3a1ba01de/volumes"
Sep 05 06:53:48.156891 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:48.156836    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="887009be-d058-4906-8359-4d6c8f5127ff" path="/var/lib/kubelet/pods/887009be-d058-4906-8359-4d6c8f5127ff/volumes"
Sep 05 06:53:54.075994 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:54.075950    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:54.140145 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:54.140099    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:54.354205 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:54.354121    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:53:58.237430 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:53:58.237387    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:00.165770 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:00.165734    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:08.443823 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:08.443784    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:14.102794 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.102760    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"]
Sep 05 06:54:14.103363 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.103009    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx" podUID="48bab18b-fd0f-4cde-9843-7a7a9bfa1449" containerName="maas-api" containerID="cri-o://a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40" gracePeriod=30
Sep 05 06:54:14.144768 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.144679    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:14.180922 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.180891    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"]
Sep 05 06:54:14.181108 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.181084    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g" podUID="eb301d82-4ec9-4007-88cc-9290f42d267a" containerName="payload-processing" containerID="cri-o://a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9" gracePeriod=30
Sep 05 06:54:14.249236 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.249201    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:14.347652 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.347526    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:14.439123 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.439095    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:54:14.447654 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.447625    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:14.468101 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.468075    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:54:14.511351 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.511317    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2wjp\" (UniqueName: \"kubernetes.io/projected/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-kube-api-access-h2wjp\") pod \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") "
Sep 05 06:54:14.511548 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.511357    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-maas-api-tls\") pod \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") "
Sep 05 06:54:14.511548 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.511386    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-openshift-service-ca\") pod \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") "
Sep 05 06:54:14.511548 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.511419    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb301d82-4ec9-4007-88cc-9290f42d267a-plugins-config-volume\") pod \"eb301d82-4ec9-4007-88cc-9290f42d267a\" (UID: \"eb301d82-4ec9-4007-88cc-9290f42d267a\") "
Sep 05 06:54:14.511548 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.511469    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq9j8\" (UniqueName: \"kubernetes.io/projected/eb301d82-4ec9-4007-88cc-9290f42d267a-kube-api-access-zq9j8\") pod \"eb301d82-4ec9-4007-88cc-9290f42d267a\" (UID: \"eb301d82-4ec9-4007-88cc-9290f42d267a\") "
Sep 05 06:54:14.511548 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.511506    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-metrics-certs\") pod \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\" (UID: \"48bab18b-fd0f-4cde-9843-7a7a9bfa1449\") "
Sep 05 06:54:14.511845 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.511815    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "48bab18b-fd0f-4cde-9843-7a7a9bfa1449" (UID: "48bab18b-fd0f-4cde-9843-7a7a9bfa1449"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:54:14.511944 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.511909    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb301d82-4ec9-4007-88cc-9290f42d267a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb301d82-4ec9-4007-88cc-9290f42d267a" (UID: "eb301d82-4ec9-4007-88cc-9290f42d267a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:54:14.513728 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.513697    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb301d82-4ec9-4007-88cc-9290f42d267a-kube-api-access-zq9j8" (OuterVolumeSpecName: "kube-api-access-zq9j8") pod "eb301d82-4ec9-4007-88cc-9290f42d267a" (UID: "eb301d82-4ec9-4007-88cc-9290f42d267a"). InnerVolumeSpecName "kube-api-access-zq9j8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:54:14.513728 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.513698    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "48bab18b-fd0f-4cde-9843-7a7a9bfa1449" (UID: "48bab18b-fd0f-4cde-9843-7a7a9bfa1449"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:54:14.513966 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.513748    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-kube-api-access-h2wjp" (OuterVolumeSpecName: "kube-api-access-h2wjp") pod "48bab18b-fd0f-4cde-9843-7a7a9bfa1449" (UID: "48bab18b-fd0f-4cde-9843-7a7a9bfa1449"). InnerVolumeSpecName "kube-api-access-h2wjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:54:14.513966 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.513883    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "48bab18b-fd0f-4cde-9843-7a7a9bfa1449" (UID: "48bab18b-fd0f-4cde-9843-7a7a9bfa1449"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:54:14.561763 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.561729    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:14.613090 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.613057    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2wjp\" (UniqueName: \"kubernetes.io/projected/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-kube-api-access-h2wjp\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:54:14.613090 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.613082    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:54:14.613090 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.613093    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:54:14.613321 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.613102    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb301d82-4ec9-4007-88cc-9290f42d267a-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:54:14.613321 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.613110    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq9j8\" (UniqueName: \"kubernetes.io/projected/eb301d82-4ec9-4007-88cc-9290f42d267a-kube-api-access-zq9j8\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:54:14.613321 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.613118    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48bab18b-fd0f-4cde-9843-7a7a9bfa1449-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:54:14.685384 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.685300    2573 generic.go:356] "Generic (PLEG): container finished" podID="eb301d82-4ec9-4007-88cc-9290f42d267a" containerID="a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9" exitCode=0
Sep 05 06:54:14.685384 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.685350    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"
Sep 05 06:54:14.685595 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.685389    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g" event={"ID":"eb301d82-4ec9-4007-88cc-9290f42d267a","Type":"ContainerDied","Data":"a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9"}
Sep 05 06:54:14.685595 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.685457    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g" event={"ID":"eb301d82-4ec9-4007-88cc-9290f42d267a","Type":"ContainerDied","Data":"51d4f15d56eb9ba5012ce07d6abab0e830167a837bb07600c1f31b7aa2669c5b"}
Sep 05 06:54:14.685595 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.685488    2573 scope.go:117] "RemoveContainer" containerID="a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9"
Sep 05 06:54:14.686785 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.686762    2573 generic.go:356] "Generic (PLEG): container finished" podID="48bab18b-fd0f-4cde-9843-7a7a9bfa1449" containerID="a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40" exitCode=0
Sep 05 06:54:14.686919 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.686809    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"
Sep 05 06:54:14.686919 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.686881    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx" event={"ID":"48bab18b-fd0f-4cde-9843-7a7a9bfa1449","Type":"ContainerDied","Data":"a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40"}
Sep 05 06:54:14.687021 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.686926    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx" event={"ID":"48bab18b-fd0f-4cde-9843-7a7a9bfa1449","Type":"ContainerDied","Data":"de972b41b857ce8386d02a5ce39812134dcb650546f743824cae97828b89b8ea"}
Sep 05 06:54:14.694379 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.694361    2573 scope.go:117] "RemoveContainer" containerID="a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9"
Sep 05 06:54:14.694665 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:54:14.694640    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9\": container with ID starting with a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9 not found: ID does not exist" containerID="a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9"
Sep 05 06:54:14.694725 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.694673    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9"} err="failed to get container status \"a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9\": rpc error: code = NotFound desc = could not find container \"a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9\": container with ID starting with a8a82f334b861518f5b2b1e0ed32d46b7e5133c31742a93c0a4546606f5d9fc9 not found: ID does not exist"
Sep 05 06:54:14.694725 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.694697    2573 scope.go:117] "RemoveContainer" containerID="a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40"
Sep 05 06:54:14.703112 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.703098    2573 scope.go:117] "RemoveContainer" containerID="a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40"
Sep 05 06:54:14.703365 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:54:14.703346    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40\": container with ID starting with a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40 not found: ID does not exist" containerID="a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40"
Sep 05 06:54:14.703425 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.703369    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40"} err="failed to get container status \"a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40\": rpc error: code = NotFound desc = could not find container \"a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40\": container with ID starting with a4f27ba22e7c903d4340ab30a7898e753530e765bb40d461dc531557cd73bc40 not found: ID does not exist"
Sep 05 06:54:14.719234 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.719209    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"]
Sep 05 06:54:14.722961 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.722941    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d36c73-5558c54f89-gjbfx"]
Sep 05 06:54:14.736719 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.736691    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"]
Sep 05 06:54:14.738561 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.738542    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d36c73-5774647c97-n928g"]
Sep 05 06:54:14.885960 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:14.885928    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:16.155876 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:16.155830    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48bab18b-fd0f-4cde-9843-7a7a9bfa1449" path="/var/lib/kubelet/pods/48bab18b-fd0f-4cde-9843-7a7a9bfa1449/volumes"
Sep 05 06:54:16.156277 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:16.156230    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb301d82-4ec9-4007-88cc-9290f42d267a" path="/var/lib/kubelet/pods/eb301d82-4ec9-4007-88cc-9290f42d267a/volumes"
Sep 05 06:54:19.886493 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:19.886461    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:24.980806 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:24.980767    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:28.687594 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:28.687555    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:56.119497 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:56.119457    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:54:57.229904 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:54:57.229847    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:03.201946 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:03.201910    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:03.275534 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:03.275496    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:03.485466 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:03.485387    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:04.039234 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:04.039198    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:14.485694 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.485655    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"]
Sep 05 06:55:14.486310 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.485961    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7" podUID="8d684c23-d8e8-47e0-9fca-892a5fa981c5" containerName="maas-api" containerID="cri-o://bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9" gracePeriod=30
Sep 05 06:55:14.516461 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.516430    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:14.599508 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.599478    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:14.666325 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.666287    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:14.738177 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.738122    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:55:14.746081 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.746060    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:14.799551 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.799523    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d684c23-d8e8-47e0-9fca-892a5fa981c5-openshift-service-ca\") pod \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") "
Sep 05 06:55:14.799728 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.799585    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls\") pod \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") "
Sep 05 06:55:14.799728 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.799606    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-metrics-certs\") pod \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") "
Sep 05 06:55:14.799728 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.799646    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vpfz\" (UniqueName: \"kubernetes.io/projected/8d684c23-d8e8-47e0-9fca-892a5fa981c5-kube-api-access-6vpfz\") pod \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\" (UID: \"8d684c23-d8e8-47e0-9fca-892a5fa981c5\") "
Sep 05 06:55:14.799978 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.799954    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d684c23-d8e8-47e0-9fca-892a5fa981c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8d684c23-d8e8-47e0-9fca-892a5fa981c5" (UID: "8d684c23-d8e8-47e0-9fca-892a5fa981c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:55:14.801738 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.801708    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8d684c23-d8e8-47e0-9fca-892a5fa981c5" (UID: "8d684c23-d8e8-47e0-9fca-892a5fa981c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:55:14.801841 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.801745    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8d684c23-d8e8-47e0-9fca-892a5fa981c5" (UID: "8d684c23-d8e8-47e0-9fca-892a5fa981c5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:55:14.801841 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.801802    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d684c23-d8e8-47e0-9fca-892a5fa981c5-kube-api-access-6vpfz" (OuterVolumeSpecName: "kube-api-access-6vpfz") pod "8d684c23-d8e8-47e0-9fca-892a5fa981c5" (UID: "8d684c23-d8e8-47e0-9fca-892a5fa981c5"). InnerVolumeSpecName "kube-api-access-6vpfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:55:14.805547 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.805524    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:14.900917 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.900887    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d684c23-d8e8-47e0-9fca-892a5fa981c5-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:14.900917 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.900915    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:14.900917 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.900924    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d684c23-d8e8-47e0-9fca-892a5fa981c5-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:14.901142 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.900931    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vpfz\" (UniqueName: \"kubernetes.io/projected/8d684c23-d8e8-47e0-9fca-892a5fa981c5-kube-api-access-6vpfz\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:14.983192 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.983162    2573 generic.go:356] "Generic (PLEG): container finished" podID="8d684c23-d8e8-47e0-9fca-892a5fa981c5" containerID="bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9" exitCode=0
Sep 05 06:55:14.983364 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.983200    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7" event={"ID":"8d684c23-d8e8-47e0-9fca-892a5fa981c5","Type":"ContainerDied","Data":"bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9"}
Sep 05 06:55:14.983364 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.983207    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"
Sep 05 06:55:14.983364 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.983238    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7" event={"ID":"8d684c23-d8e8-47e0-9fca-892a5fa981c5","Type":"ContainerDied","Data":"77060ebed0c506f7724913bdb8a8e527d83798ff83db2e05ab03830f7198c373"}
Sep 05 06:55:14.983364 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.983259    2573 scope.go:117] "RemoveContainer" containerID="bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9"
Sep 05 06:55:14.996269 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.996248    2573 scope.go:117] "RemoveContainer" containerID="bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9"
Sep 05 06:55:14.996589 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:55:14.996566    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9\": container with ID starting with bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9 not found: ID does not exist" containerID="bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9"
Sep 05 06:55:14.996641 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:14.996600    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9"} err="failed to get container status \"bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9\": rpc error: code = NotFound desc = could not find container \"bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9\": container with ID starting with bfe6aef4d5cfa79c4d40e0233fa03188c93becf03f2b14f249e6719bd17e04b9 not found: ID does not exist"
Sep 05 06:55:15.009096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:15.009074    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"]
Sep 05 06:55:15.014000 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:15.013978    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b49086-85546bd879-94wp7"]
Sep 05 06:55:15.124641 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:15.124609    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:16.156672 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:16.156635    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d684c23-d8e8-47e0-9fca-892a5fa981c5" path="/var/lib/kubelet/pods/8d684c23-d8e8-47e0-9fca-892a5fa981c5/volumes"
Sep 05 06:55:21.411294 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:21.411252    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:21.511572 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:21.511534    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:21.681742 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:21.681637    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:22.146409 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:22.146335    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:32.526575 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.526541    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"]
Sep 05 06:55:32.527106 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.526785    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w" podUID="62500340-a383-45b4-8dbb-e3f9e02e292f" containerName="maas-api" containerID="cri-o://a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742" gracePeriod=30
Sep 05 06:55:32.563886 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.563842    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:32.627603 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.627562    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:32.727991 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.727958    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:32.773311 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.773283    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:55:32.795420 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.795301    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:32.876934 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.876906    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-maas-api-tls\") pod \"62500340-a383-45b4-8dbb-e3f9e02e292f\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") "
Sep 05 06:55:32.877100 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.876951    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5llr\" (UniqueName: \"kubernetes.io/projected/62500340-a383-45b4-8dbb-e3f9e02e292f-kube-api-access-w5llr\") pod \"62500340-a383-45b4-8dbb-e3f9e02e292f\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") "
Sep 05 06:55:32.877100 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.877036    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-metrics-certs\") pod \"62500340-a383-45b4-8dbb-e3f9e02e292f\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") "
Sep 05 06:55:32.877100 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.877054    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62500340-a383-45b4-8dbb-e3f9e02e292f-openshift-service-ca\") pod \"62500340-a383-45b4-8dbb-e3f9e02e292f\" (UID: \"62500340-a383-45b4-8dbb-e3f9e02e292f\") "
Sep 05 06:55:32.877483 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.877457    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62500340-a383-45b4-8dbb-e3f9e02e292f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "62500340-a383-45b4-8dbb-e3f9e02e292f" (UID: "62500340-a383-45b4-8dbb-e3f9e02e292f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:55:32.879128 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.879103    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "62500340-a383-45b4-8dbb-e3f9e02e292f" (UID: "62500340-a383-45b4-8dbb-e3f9e02e292f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:55:32.879236 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.879146    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "62500340-a383-45b4-8dbb-e3f9e02e292f" (UID: "62500340-a383-45b4-8dbb-e3f9e02e292f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:55:32.879236 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.879165    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62500340-a383-45b4-8dbb-e3f9e02e292f-kube-api-access-w5llr" (OuterVolumeSpecName: "kube-api-access-w5llr") pod "62500340-a383-45b4-8dbb-e3f9e02e292f" (UID: "62500340-a383-45b4-8dbb-e3f9e02e292f"). InnerVolumeSpecName "kube-api-access-w5llr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:55:32.977905 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.977844    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:32.978084 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.977911    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62500340-a383-45b4-8dbb-e3f9e02e292f-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:32.978084 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.977926    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62500340-a383-45b4-8dbb-e3f9e02e292f-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:32.978084 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:32.977939    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5llr\" (UniqueName: \"kubernetes.io/projected/62500340-a383-45b4-8dbb-e3f9e02e292f-kube-api-access-w5llr\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:33.073468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:33.073384    2573 generic.go:356] "Generic (PLEG): container finished" podID="62500340-a383-45b4-8dbb-e3f9e02e292f" containerID="a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742" exitCode=0
Sep 05 06:55:33.073468 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:33.073427    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"
Sep 05 06:55:33.073670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:33.073459    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w" event={"ID":"62500340-a383-45b4-8dbb-e3f9e02e292f","Type":"ContainerDied","Data":"a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742"}
Sep 05 06:55:33.073670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:33.073516    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w" event={"ID":"62500340-a383-45b4-8dbb-e3f9e02e292f","Type":"ContainerDied","Data":"f45068577c78fea34282fca50f04e0b34f26206c6402d0bceb0e3a96e334d4ca"}
Sep 05 06:55:33.073670 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:33.073537    2573 scope.go:117] "RemoveContainer" containerID="a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742"
Sep 05 06:55:33.082867 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:33.082829    2573 scope.go:117] "RemoveContainer" containerID="a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742"
Sep 05 06:55:33.083175 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:55:33.083157    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742\": container with ID starting with a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742 not found: ID does not exist" containerID="a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742"
Sep 05 06:55:33.083243 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:33.083188    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742"} err="failed to get container status \"a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742\": rpc error: code = NotFound desc = could not find container \"a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742\": container with ID starting with a1e651615047c253b9b502c18395ecd81f58f99a4de9bd8edeaaaf501b0a5742 not found: ID does not exist"
Sep 05 06:55:33.096502 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:33.096481    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"]
Sep 05 06:55:33.100337 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:33.100320    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-183ae6-78654c7fd6-2qj5w"]
Sep 05 06:55:33.146236 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:33.146196    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:34.156065 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:34.156031    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62500340-a383-45b4-8dbb-e3f9e02e292f" path="/var/lib/kubelet/pods/62500340-a383-45b4-8dbb-e3f9e02e292f/volumes"
Sep 05 06:55:39.494240 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:39.494212    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:39.548907 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:39.548876    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:39.755972 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:39.755890    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:40.332339 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:40.332303    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:47.967965 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:47.967829    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:50.216318 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:50.216281    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:50.845527 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:50.845492    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"]
Sep 05 06:55:50.845919 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:50.845736    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2" podUID="823aa414-5ccc-468c-95d7-6207de93926d" containerName="maas-api" containerID="cri-o://19cec060e539fbf7eb473106ea956133ea87df39631b98d32524e265e576fa83" gracePeriod=30
Sep 05 06:55:50.871609 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:50.871580    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:50.904806 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:50.904778    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"]
Sep 05 06:55:50.905319 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:50.905057    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl" podUID="1f999151-f340-4c43-a201-536b610b9048" containerName="payload-processing" containerID="cri-o://41f87e921cd62b0dcefeea80e1af7814aea908b2ad83755a4085feb34150eb79" gracePeriod=30
Sep 05 06:55:50.910493 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:50.910460    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"]
Sep 05 06:55:50.910709 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:50.910678    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4" podUID="5082c60e-3dc9-497e-a709-07cf4bcfd827" containerName="payload-pre-processing" containerID="cri-o://7ea4958b6379da4bb1085151e5f74f6178489226b95866dd1e6bf06b4763986b" gracePeriod=30
Sep 05 06:55:50.941321 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:50.941271    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:51.004044 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.004015    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:51.091837 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.091791    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:51.156580 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.156258    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:51.163373 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.163345    2573 generic.go:356] "Generic (PLEG): container finished" podID="1f999151-f340-4c43-a201-536b610b9048" containerID="41f87e921cd62b0dcefeea80e1af7814aea908b2ad83755a4085feb34150eb79" exitCode=0
Sep 05 06:55:51.163487 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.163416    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl" event={"ID":"1f999151-f340-4c43-a201-536b610b9048","Type":"ContainerDied","Data":"41f87e921cd62b0dcefeea80e1af7814aea908b2ad83755a4085feb34150eb79"}
Sep 05 06:55:51.164913 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.164895    2573 generic.go:356] "Generic (PLEG): container finished" podID="823aa414-5ccc-468c-95d7-6207de93926d" containerID="19cec060e539fbf7eb473106ea956133ea87df39631b98d32524e265e576fa83" exitCode=0
Sep 05 06:55:51.165031 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.164962    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2" event={"ID":"823aa414-5ccc-468c-95d7-6207de93926d","Type":"ContainerDied","Data":"19cec060e539fbf7eb473106ea956133ea87df39631b98d32524e265e576fa83"}
Sep 05 06:55:51.165031 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.165001    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2" event={"ID":"823aa414-5ccc-468c-95d7-6207de93926d","Type":"ContainerDied","Data":"ae872680133dbd85cd5bcebd1ed59a4f95d18edf7718e6ec70a06b9c6c3352c9"}
Sep 05 06:55:51.165031 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.165023    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae872680133dbd85cd5bcebd1ed59a4f95d18edf7718e6ec70a06b9c6c3352c9"
Sep 05 06:55:51.166646 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.166626    2573 generic.go:356] "Generic (PLEG): container finished" podID="5082c60e-3dc9-497e-a709-07cf4bcfd827" containerID="7ea4958b6379da4bb1085151e5f74f6178489226b95866dd1e6bf06b4763986b" exitCode=0
Sep 05 06:55:51.166743 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.166706    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4" event={"ID":"5082c60e-3dc9-497e-a709-07cf4bcfd827","Type":"ContainerDied","Data":"7ea4958b6379da4bb1085151e5f74f6178489226b95866dd1e6bf06b4763986b"}
Sep 05 06:55:51.168094 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.168074    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:55:51.179685 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.179663    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:55:51.192844 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.192822    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:55:51.249587 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.249562    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz2kq\" (UniqueName: \"kubernetes.io/projected/823aa414-5ccc-468c-95d7-6207de93926d-kube-api-access-bz2kq\") pod \"823aa414-5ccc-468c-95d7-6207de93926d\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") "
Sep 05 06:55:51.249998 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.249610    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rzcp\" (UniqueName: \"kubernetes.io/projected/1f999151-f340-4c43-a201-536b610b9048-kube-api-access-5rzcp\") pod \"1f999151-f340-4c43-a201-536b610b9048\" (UID: \"1f999151-f340-4c43-a201-536b610b9048\") "
Sep 05 06:55:51.249998 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.249680    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-metrics-certs\") pod \"823aa414-5ccc-468c-95d7-6207de93926d\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") "
Sep 05 06:55:51.249998 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.249722    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f999151-f340-4c43-a201-536b610b9048-plugins-config-volume\") pod \"1f999151-f340-4c43-a201-536b610b9048\" (UID: \"1f999151-f340-4c43-a201-536b610b9048\") "
Sep 05 06:55:51.249998 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.249838    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/823aa414-5ccc-468c-95d7-6207de93926d-openshift-service-ca\") pod \"823aa414-5ccc-468c-95d7-6207de93926d\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") "
Sep 05 06:55:51.249998 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.249903    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-maas-api-tls\") pod \"823aa414-5ccc-468c-95d7-6207de93926d\" (UID: \"823aa414-5ccc-468c-95d7-6207de93926d\") "
Sep 05 06:55:51.250267 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.250115    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f999151-f340-4c43-a201-536b610b9048-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f999151-f340-4c43-a201-536b610b9048" (UID: "1f999151-f340-4c43-a201-536b610b9048"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:55:51.250267 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.250201    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/823aa414-5ccc-468c-95d7-6207de93926d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "823aa414-5ccc-468c-95d7-6207de93926d" (UID: "823aa414-5ccc-468c-95d7-6207de93926d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:55:51.250386 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.250325    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/823aa414-5ccc-468c-95d7-6207de93926d-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:51.250386 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.250344    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f999151-f340-4c43-a201-536b610b9048-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:51.251759 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.251734    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f999151-f340-4c43-a201-536b610b9048-kube-api-access-5rzcp" (OuterVolumeSpecName: "kube-api-access-5rzcp") pod "1f999151-f340-4c43-a201-536b610b9048" (UID: "1f999151-f340-4c43-a201-536b610b9048"). InnerVolumeSpecName "kube-api-access-5rzcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:55:51.251943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.251826    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/823aa414-5ccc-468c-95d7-6207de93926d-kube-api-access-bz2kq" (OuterVolumeSpecName: "kube-api-access-bz2kq") pod "823aa414-5ccc-468c-95d7-6207de93926d" (UID: "823aa414-5ccc-468c-95d7-6207de93926d"). InnerVolumeSpecName "kube-api-access-bz2kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:55:51.252061 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.252043    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "823aa414-5ccc-468c-95d7-6207de93926d" (UID: "823aa414-5ccc-468c-95d7-6207de93926d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:55:51.252173 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.252156    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "823aa414-5ccc-468c-95d7-6207de93926d" (UID: "823aa414-5ccc-468c-95d7-6207de93926d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:55:51.351622 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.351589    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5082c60e-3dc9-497e-a709-07cf4bcfd827-plugins-config-volume\") pod \"5082c60e-3dc9-497e-a709-07cf4bcfd827\" (UID: \"5082c60e-3dc9-497e-a709-07cf4bcfd827\") "
Sep 05 06:55:51.351762 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.351692    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbr4b\" (UniqueName: \"kubernetes.io/projected/5082c60e-3dc9-497e-a709-07cf4bcfd827-kube-api-access-cbr4b\") pod \"5082c60e-3dc9-497e-a709-07cf4bcfd827\" (UID: \"5082c60e-3dc9-497e-a709-07cf4bcfd827\") "
Sep 05 06:55:51.352032 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.352014    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:51.352096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.352042    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/823aa414-5ccc-468c-95d7-6207de93926d-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:51.352096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.352056    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz2kq\" (UniqueName: \"kubernetes.io/projected/823aa414-5ccc-468c-95d7-6207de93926d-kube-api-access-bz2kq\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:51.352096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.352066    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rzcp\" (UniqueName: \"kubernetes.io/projected/1f999151-f340-4c43-a201-536b610b9048-kube-api-access-5rzcp\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:51.352096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.352015    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5082c60e-3dc9-497e-a709-07cf4bcfd827-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5082c60e-3dc9-497e-a709-07cf4bcfd827" (UID: "5082c60e-3dc9-497e-a709-07cf4bcfd827"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:55:51.353541 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.353520    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5082c60e-3dc9-497e-a709-07cf4bcfd827-kube-api-access-cbr4b" (OuterVolumeSpecName: "kube-api-access-cbr4b") pod "5082c60e-3dc9-497e-a709-07cf4bcfd827" (UID: "5082c60e-3dc9-497e-a709-07cf4bcfd827"). InnerVolumeSpecName "kube-api-access-cbr4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:55:51.452945 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.452847    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbr4b\" (UniqueName: \"kubernetes.io/projected/5082c60e-3dc9-497e-a709-07cf4bcfd827-kube-api-access-cbr4b\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:51.452945 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.452899    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5082c60e-3dc9-497e-a709-07cf4bcfd827-plugins-config-volume\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:55:51.509422 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:51.509384    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:52.175031 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.175006    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"
Sep 05 06:55:52.175149 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.175028    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl" event={"ID":"1f999151-f340-4c43-a201-536b610b9048","Type":"ContainerDied","Data":"3a8e678a8489daed30a4182e99746cec101b2a8afe3bd3ab36e2c03895b3087d"}
Sep 05 06:55:52.175149 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.175095    2573 scope.go:117] "RemoveContainer" containerID="41f87e921cd62b0dcefeea80e1af7814aea908b2ad83755a4085feb34150eb79"
Sep 05 06:55:52.176452 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.176425    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4" event={"ID":"5082c60e-3dc9-497e-a709-07cf4bcfd827","Type":"ContainerDied","Data":"7463333cb666ccda6abcf9a106b4e947b0158f7ce1fbfb08ae7106fecccf4bd8"}
Sep 05 06:55:52.176558 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.176454    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"
Sep 05 06:55:52.176558 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.176463    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"
Sep 05 06:55:52.186686 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.186667    2573 scope.go:117] "RemoveContainer" containerID="7ea4958b6379da4bb1085151e5f74f6178489226b95866dd1e6bf06b4763986b"
Sep 05 06:55:52.198968 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.198947    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"]
Sep 05 06:55:52.203730 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.203709    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-39047b-64d667f67d-4pmt4"]
Sep 05 06:55:52.217050 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.217027    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"]
Sep 05 06:55:52.223071 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.223047    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-39047b-858dc9b9c9-z52vl"]
Sep 05 06:55:52.224837 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.224819    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:55:52.230168 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.230148    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 06:55:52.234007 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.233989    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"]
Sep 05 06:55:52.239881 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.239843    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-39047b-dccdd6b84-mvxk2"]
Sep 05 06:55:52.433556 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:52.433477    2573 scope.go:117] "RemoveContainer" containerID="19cec060e539fbf7eb473106ea956133ea87df39631b98d32524e265e576fa83"
Sep 05 06:55:54.155088 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:54.155058    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f999151-f340-4c43-a201-536b610b9048" path="/var/lib/kubelet/pods/1f999151-f340-4c43-a201-536b610b9048/volumes"
Sep 05 06:55:54.155464 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:54.155444    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5082c60e-3dc9-497e-a709-07cf4bcfd827" path="/var/lib/kubelet/pods/5082c60e-3dc9-497e-a709-07cf4bcfd827/volumes"
Sep 05 06:55:54.155796 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:54.155784    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="823aa414-5ccc-468c-95d7-6207de93926d" path="/var/lib/kubelet/pods/823aa414-5ccc-468c-95d7-6207de93926d/volumes"
Sep 05 06:55:57.791765 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:57.791729    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:57.891925 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:57.891891    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:55:58.060466 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:55:58.060377    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:56:00.557933 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:00.557904    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-66bwj"]
Sep 05 06:56:00.558365 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:00.558108    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6bcd5999b5-66bwj" podUID="ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a" containerName="manager" containerID="cri-o://d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de" gracePeriod=30
Sep 05 06:56:00.802050 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:00.802027    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:56:00.947491 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:00.947464    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-webhook-cert\") pod \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") "
Sep 05 06:56:00.947662 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:00.947520    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5dvv\" (UniqueName: \"kubernetes.io/projected/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-kube-api-access-t5dvv\") pod \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") "
Sep 05 06:56:00.947662 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:00.947607    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-metrics-certs\") pod \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\" (UID: \"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a\") "
Sep 05 06:56:00.949529 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:00.949502    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a" (UID: "ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:56:00.949669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:00.949587    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-kube-api-access-t5dvv" (OuterVolumeSpecName: "kube-api-access-t5dvv") pod "ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a" (UID: "ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a"). InnerVolumeSpecName "kube-api-access-t5dvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:56:00.949669 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:00.949632    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a" (UID: "ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:56:01.049303 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.049190    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5dvv\" (UniqueName: \"kubernetes.io/projected/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-kube-api-access-t5dvv\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:56:01.049303 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.049236    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:56:01.049303 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.049246    2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a-webhook-cert\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:56:01.218940 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.218909    2573 generic.go:356] "Generic (PLEG): container finished" podID="ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a" containerID="d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de" exitCode=0
Sep 05 06:56:01.219111 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.218954    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcd5999b5-66bwj"
Sep 05 06:56:01.219111 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.218974    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcd5999b5-66bwj" event={"ID":"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a","Type":"ContainerDied","Data":"d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de"}
Sep 05 06:56:01.219111 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.219014    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcd5999b5-66bwj" event={"ID":"ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a","Type":"ContainerDied","Data":"84ec2f742a7dea8ebefb84b9119a8febd4fc80ddf7f36fc1b7d96615ded4fb9d"}
Sep 05 06:56:01.219111 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.219036    2573 scope.go:117] "RemoveContainer" containerID="d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de"
Sep 05 06:56:01.228742 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.228724    2573 scope.go:117] "RemoveContainer" containerID="d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de"
Sep 05 06:56:01.229041 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:56:01.229023    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de\": container with ID starting with d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de not found: ID does not exist" containerID="d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de"
Sep 05 06:56:01.229116 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.229052    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de"} err="failed to get container status \"d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de\": rpc error: code = NotFound desc = could not find container \"d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de\": container with ID starting with d1e2036ab3c2685b2f088e4179c9e8bcbeac5dcbb3b69caa62009b89ea1082de not found: ID does not exist"
Sep 05 06:56:01.242074 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.242050    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-66bwj"]
Sep 05 06:56:01.247722 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.247702    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-66bwj"]
Sep 05 06:56:01.623511 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:01.623482    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:56:02.161779 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:02.161744    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a" path="/var/lib/kubelet/pods/ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a/volumes"
Sep 05 06:56:51.984656 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:56:51.984619    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:57:02.052643 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.052604    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-g2jkg"]
Sep 05 06:57:02.053060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.052984    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62500340-a383-45b4-8dbb-e3f9e02e292f" containerName="maas-api"
Sep 05 06:57:02.053060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.052995    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="62500340-a383-45b4-8dbb-e3f9e02e292f" containerName="maas-api"
Sep 05 06:57:02.053060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053003    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48bab18b-fd0f-4cde-9843-7a7a9bfa1449" containerName="maas-api"
Sep 05 06:57:02.053060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053009    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="48bab18b-fd0f-4cde-9843-7a7a9bfa1449" containerName="maas-api"
Sep 05 06:57:02.053060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053017    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a" containerName="manager"
Sep 05 06:57:02.053060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053022    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a" containerName="manager"
Sep 05 06:57:02.053060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053032    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5082c60e-3dc9-497e-a709-07cf4bcfd827" containerName="payload-pre-processing"
Sep 05 06:57:02.053060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053037    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5082c60e-3dc9-497e-a709-07cf4bcfd827" containerName="payload-pre-processing"
Sep 05 06:57:02.053060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053046    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d684c23-d8e8-47e0-9fca-892a5fa981c5" containerName="maas-api"
Sep 05 06:57:02.053060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053051    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d684c23-d8e8-47e0-9fca-892a5fa981c5" containerName="maas-api"
Sep 05 06:57:02.053060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053062    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f999151-f340-4c43-a201-536b610b9048" containerName="payload-processing"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053069    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f999151-f340-4c43-a201-536b610b9048" containerName="payload-processing"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053084    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="823aa414-5ccc-468c-95d7-6207de93926d" containerName="maas-api"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053089    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="823aa414-5ccc-468c-95d7-6207de93926d" containerName="maas-api"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053094    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6607cfc3-9233-4188-843c-c6b3a1ba01de" containerName="maas-api"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053099    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6607cfc3-9233-4188-843c-c6b3a1ba01de" containerName="maas-api"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053105    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="887009be-d058-4906-8359-4d6c8f5127ff" containerName="payload-pre-processing"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053110    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="887009be-d058-4906-8359-4d6c8f5127ff" containerName="payload-pre-processing"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053119    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb301d82-4ec9-4007-88cc-9290f42d267a" containerName="payload-processing"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053124    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb301d82-4ec9-4007-88cc-9290f42d267a" containerName="payload-processing"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053171    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6607cfc3-9233-4188-843c-c6b3a1ba01de" containerName="maas-api"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053180    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="48bab18b-fd0f-4cde-9843-7a7a9bfa1449" containerName="maas-api"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053186    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5082c60e-3dc9-497e-a709-07cf4bcfd827" containerName="payload-pre-processing"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053194    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb301d82-4ec9-4007-88cc-9290f42d267a" containerName="payload-processing"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053200    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f999151-f340-4c43-a201-536b610b9048" containerName="payload-processing"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053213    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d684c23-d8e8-47e0-9fca-892a5fa981c5" containerName="maas-api"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053220    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="887009be-d058-4906-8359-4d6c8f5127ff" containerName="payload-pre-processing"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053227    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="62500340-a383-45b4-8dbb-e3f9e02e292f" containerName="maas-api"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053231    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad2ceaaa-b8a4-4e08-adf6-2f9eff78ab6a" containerName="manager"
Sep 05 06:57:02.053450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.053236    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="823aa414-5ccc-468c-95d7-6207de93926d" containerName="maas-api"
Sep 05 06:57:02.056269 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.056254    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:02.059748 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.059721    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 05 06:57:02.061004 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.060981    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 05 06:57:02.061110 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.061009    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 05 06:57:02.061110 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.061092    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4zf7f\""
Sep 05 06:57:02.067579 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.067559    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-g2jkg"]
Sep 05 06:57:02.096060 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.096034    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g7zq\" (UniqueName: \"kubernetes.io/projected/8173e75c-53d6-4050-a6e4-21bbf16241c1-kube-api-access-5g7zq\") pod \"maas-controller-6bcd5999b5-g2jkg\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") " pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:02.096171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.096136    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-webhook-cert\") pod \"maas-controller-6bcd5999b5-g2jkg\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") " pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:02.096171 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.096163    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-metrics-certs\") pod \"maas-controller-6bcd5999b5-g2jkg\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") " pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:02.196908 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.196881    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-webhook-cert\") pod \"maas-controller-6bcd5999b5-g2jkg\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") " pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:02.196908 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.196909    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-metrics-certs\") pod \"maas-controller-6bcd5999b5-g2jkg\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") " pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:02.197081 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.196956    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g7zq\" (UniqueName: \"kubernetes.io/projected/8173e75c-53d6-4050-a6e4-21bbf16241c1-kube-api-access-5g7zq\") pod \"maas-controller-6bcd5999b5-g2jkg\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") " pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:02.199413 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.199395    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-webhook-cert\") pod \"maas-controller-6bcd5999b5-g2jkg\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") " pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:02.199471 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.199412    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-metrics-certs\") pod \"maas-controller-6bcd5999b5-g2jkg\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") " pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:02.207223 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.207203    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g7zq\" (UniqueName: \"kubernetes.io/projected/8173e75c-53d6-4050-a6e4-21bbf16241c1-kube-api-access-5g7zq\") pod \"maas-controller-6bcd5999b5-g2jkg\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") " pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:02.366587 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.366507    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:02.637996 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.637916    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-g2jkg"]
Sep 05 06:57:02.641186 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:57:02.641148    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8173e75c_53d6_4050_a6e4_21bbf16241c1.slice/crio-f5d524b5c71435cf2d72debc0cb89ed989e64239e13935e937c68fbccd2d2f48 WatchSource:0}: Error finding container f5d524b5c71435cf2d72debc0cb89ed989e64239e13935e937c68fbccd2d2f48: Status 404 returned error can't find the container with id f5d524b5c71435cf2d72debc0cb89ed989e64239e13935e937c68fbccd2d2f48
Sep 05 06:57:02.642494 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:02.642474    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 06:57:03.517110 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:03.517073    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg" event={"ID":"8173e75c-53d6-4050-a6e4-21bbf16241c1","Type":"ContainerStarted","Data":"6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4"}
Sep 05 06:57:03.517512 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:03.517135    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg" event={"ID":"8173e75c-53d6-4050-a6e4-21bbf16241c1","Type":"ContainerStarted","Data":"f5d524b5c71435cf2d72debc0cb89ed989e64239e13935e937c68fbccd2d2f48"}
Sep 05 06:57:03.517512 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:03.517189    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:03.535992 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:03.535953    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg" podStartSLOduration=1.127015546 podStartE2EDuration="1.535938015s" totalImagesPullingTime="408.922469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:57:03.534665629 +0000 UTC m=+1871.902712841" watchObservedRunningTime="2026-09-05 06:57:03.535938015 +0000 UTC m=+1871.903985218"
Sep 05 06:57:14.523208 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:14.523135    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:21.722183 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:21.722146    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:57:23.725349 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:23.725314    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:57:23.832170 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:23.832141    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-g2jkg"]
Sep 05 06:57:23.832348 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:23.832327    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg" podUID="8173e75c-53d6-4050-a6e4-21bbf16241c1" containerName="manager" containerID="cri-o://6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4" gracePeriod=30
Sep 05 06:57:24.074233 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.074207    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:24.201388 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.201357    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-metrics-certs\") pod \"8173e75c-53d6-4050-a6e4-21bbf16241c1\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") "
Sep 05 06:57:24.201550 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.201426    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-webhook-cert\") pod \"8173e75c-53d6-4050-a6e4-21bbf16241c1\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") "
Sep 05 06:57:24.201550 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.201482    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g7zq\" (UniqueName: \"kubernetes.io/projected/8173e75c-53d6-4050-a6e4-21bbf16241c1-kube-api-access-5g7zq\") pod \"8173e75c-53d6-4050-a6e4-21bbf16241c1\" (UID: \"8173e75c-53d6-4050-a6e4-21bbf16241c1\") "
Sep 05 06:57:24.203402 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.203368    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8173e75c-53d6-4050-a6e4-21bbf16241c1" (UID: "8173e75c-53d6-4050-a6e4-21bbf16241c1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:57:24.203527 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.203475    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8173e75c-53d6-4050-a6e4-21bbf16241c1-kube-api-access-5g7zq" (OuterVolumeSpecName: "kube-api-access-5g7zq") pod "8173e75c-53d6-4050-a6e4-21bbf16241c1" (UID: "8173e75c-53d6-4050-a6e4-21bbf16241c1"). InnerVolumeSpecName "kube-api-access-5g7zq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:57:24.203573 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.203529    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8173e75c-53d6-4050-a6e4-21bbf16241c1" (UID: "8173e75c-53d6-4050-a6e4-21bbf16241c1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:57:24.302920 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.302822    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g7zq\" (UniqueName: \"kubernetes.io/projected/8173e75c-53d6-4050-a6e4-21bbf16241c1-kube-api-access-5g7zq\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:57:24.302920 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.302872    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:57:24.302920 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.302883    2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8173e75c-53d6-4050-a6e4-21bbf16241c1-webhook-cert\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 06:57:24.622794 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.622704    2573 generic.go:356] "Generic (PLEG): container finished" podID="8173e75c-53d6-4050-a6e4-21bbf16241c1" containerID="6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4" exitCode=0
Sep 05 06:57:24.622794 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.622750    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg"
Sep 05 06:57:24.622794 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.622781    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg" event={"ID":"8173e75c-53d6-4050-a6e4-21bbf16241c1","Type":"ContainerDied","Data":"6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4"}
Sep 05 06:57:24.623091 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.622832    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcd5999b5-g2jkg" event={"ID":"8173e75c-53d6-4050-a6e4-21bbf16241c1","Type":"ContainerDied","Data":"f5d524b5c71435cf2d72debc0cb89ed989e64239e13935e937c68fbccd2d2f48"}
Sep 05 06:57:24.623091 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.622871    2573 scope.go:117] "RemoveContainer" containerID="6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4"
Sep 05 06:57:24.632038 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.632023    2573 scope.go:117] "RemoveContainer" containerID="6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4"
Sep 05 06:57:24.632305 ip-10-0-128-174 kubenswrapper[2573]: E0905 06:57:24.632287    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4\": container with ID starting with 6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4 not found: ID does not exist" containerID="6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4"
Sep 05 06:57:24.632350 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.632313    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4"} err="failed to get container status \"6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4\": rpc error: code = NotFound desc = could not find container \"6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4\": container with ID starting with 6cc275886aae108982caca8ca6f802a3d6f83af8b46b08a5248ef8988621b4d4 not found: ID does not exist"
Sep 05 06:57:24.645106 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.645086    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-g2jkg"]
Sep 05 06:57:24.652152 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:24.652125    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-g2jkg"]
Sep 05 06:57:25.322402 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.322361    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:57:25.432283 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.432247    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-7p5w9"]
Sep 05 06:57:25.432711 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.432693    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8173e75c-53d6-4050-a6e4-21bbf16241c1" containerName="manager"
Sep 05 06:57:25.432803 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.432712    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8173e75c-53d6-4050-a6e4-21bbf16241c1" containerName="manager"
Sep 05 06:57:25.432881 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.432801    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8173e75c-53d6-4050-a6e4-21bbf16241c1" containerName="manager"
Sep 05 06:57:25.435758 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.435737    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:25.439263 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.439242    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 05 06:57:25.439263 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.439258    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 05 06:57:25.439441 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.439264    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 05 06:57:25.439441 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.439244    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4zf7f\""
Sep 05 06:57:25.444316 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.444299    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-7p5w9"]
Sep 05 06:57:25.616160 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.616057    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/769dff76-6513-4b94-834e-7384a6c0fae5-webhook-cert\") pod \"maas-controller-6bcd5999b5-7p5w9\" (UID: \"769dff76-6513-4b94-834e-7384a6c0fae5\") " pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:25.616160 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.616118    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/769dff76-6513-4b94-834e-7384a6c0fae5-metrics-certs\") pod \"maas-controller-6bcd5999b5-7p5w9\" (UID: \"769dff76-6513-4b94-834e-7384a6c0fae5\") " pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:25.616362 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.616193    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtbg4\" (UniqueName: \"kubernetes.io/projected/769dff76-6513-4b94-834e-7384a6c0fae5-kube-api-access-xtbg4\") pod \"maas-controller-6bcd5999b5-7p5w9\" (UID: \"769dff76-6513-4b94-834e-7384a6c0fae5\") " pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:25.716659 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.716630    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/769dff76-6513-4b94-834e-7384a6c0fae5-webhook-cert\") pod \"maas-controller-6bcd5999b5-7p5w9\" (UID: \"769dff76-6513-4b94-834e-7384a6c0fae5\") " pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:25.716862 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.716685    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/769dff76-6513-4b94-834e-7384a6c0fae5-metrics-certs\") pod \"maas-controller-6bcd5999b5-7p5w9\" (UID: \"769dff76-6513-4b94-834e-7384a6c0fae5\") " pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:25.716862 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.716730    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtbg4\" (UniqueName: \"kubernetes.io/projected/769dff76-6513-4b94-834e-7384a6c0fae5-kube-api-access-xtbg4\") pod \"maas-controller-6bcd5999b5-7p5w9\" (UID: \"769dff76-6513-4b94-834e-7384a6c0fae5\") " pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:25.719222 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.719190    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/769dff76-6513-4b94-834e-7384a6c0fae5-webhook-cert\") pod \"maas-controller-6bcd5999b5-7p5w9\" (UID: \"769dff76-6513-4b94-834e-7384a6c0fae5\") " pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:25.719500 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.719478    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/769dff76-6513-4b94-834e-7384a6c0fae5-metrics-certs\") pod \"maas-controller-6bcd5999b5-7p5w9\" (UID: \"769dff76-6513-4b94-834e-7384a6c0fae5\") " pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:25.726901 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.726875    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtbg4\" (UniqueName: \"kubernetes.io/projected/769dff76-6513-4b94-834e-7384a6c0fae5-kube-api-access-xtbg4\") pod \"maas-controller-6bcd5999b5-7p5w9\" (UID: \"769dff76-6513-4b94-834e-7384a6c0fae5\") " pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:25.746279 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:25.746249    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:26.028756 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:26.028732    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bcd5999b5-7p5w9"]
Sep 05 06:57:26.030725 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:57:26.030699    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod769dff76_6513_4b94_834e_7384a6c0fae5.slice/crio-8fabadc74425b5b983cad05b2440c70a66ef26e1d023d279653621b96f8d33af WatchSource:0}: Error finding container 8fabadc74425b5b983cad05b2440c70a66ef26e1d023d279653621b96f8d33af: Status 404 returned error can't find the container with id 8fabadc74425b5b983cad05b2440c70a66ef26e1d023d279653621b96f8d33af
Sep 05 06:57:26.155973 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:26.155943    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8173e75c-53d6-4050-a6e4-21bbf16241c1" path="/var/lib/kubelet/pods/8173e75c-53d6-4050-a6e4-21bbf16241c1/volumes"
Sep 05 06:57:26.635120 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:26.635026    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcd5999b5-7p5w9" event={"ID":"769dff76-6513-4b94-834e-7384a6c0fae5","Type":"ContainerStarted","Data":"a64335339ae27d9debe7e9969199eb2b50f7973564a65a80a3b4a4a64b11c298"}
Sep 05 06:57:26.635120 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:26.635100    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcd5999b5-7p5w9" event={"ID":"769dff76-6513-4b94-834e-7384a6c0fae5","Type":"ContainerStarted","Data":"8fabadc74425b5b983cad05b2440c70a66ef26e1d023d279653621b96f8d33af"}
Sep 05 06:57:26.635526 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:26.635151    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:57:26.657636 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:26.657585    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bcd5999b5-7p5w9" podStartSLOduration=1.354152834 podStartE2EDuration="1.657568447s" totalImagesPullingTime="303.415613ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:57:26.65639366 +0000 UTC m=+1895.024440903" watchObservedRunningTime="2026-09-05 06:57:26.657568447 +0000 UTC m=+1895.025615708"
Sep 05 06:57:37.642025 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:57:37.641987    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bcd5999b5-7p5w9"
Sep 05 06:58:14.496535 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:58:14.496492    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 06:59:40.051096 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.051063    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"]
Sep 05 06:59:40.054600 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.054583    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"
Sep 05 06:59:40.057305 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.057281    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gkp9h\""
Sep 05 06:59:40.064263 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.064243    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"]
Sep 05 06:59:40.220943 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.220909    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbz9g\" (UniqueName: \"kubernetes.io/projected/47f15b0c-ffc8-427c-83e0-1dde521dbd03-kube-api-access-rbz9g\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4ncr7\" (UID: \"47f15b0c-ffc8-427c-83e0-1dde521dbd03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"
Sep 05 06:59:40.221115 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.220959    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47f15b0c-ffc8-427c-83e0-1dde521dbd03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4ncr7\" (UID: \"47f15b0c-ffc8-427c-83e0-1dde521dbd03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"
Sep 05 06:59:40.321433 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.321352    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbz9g\" (UniqueName: \"kubernetes.io/projected/47f15b0c-ffc8-427c-83e0-1dde521dbd03-kube-api-access-rbz9g\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4ncr7\" (UID: \"47f15b0c-ffc8-427c-83e0-1dde521dbd03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"
Sep 05 06:59:40.321583 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.321467    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47f15b0c-ffc8-427c-83e0-1dde521dbd03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4ncr7\" (UID: \"47f15b0c-ffc8-427c-83e0-1dde521dbd03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"
Sep 05 06:59:40.321909 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.321893    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47f15b0c-ffc8-427c-83e0-1dde521dbd03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4ncr7\" (UID: \"47f15b0c-ffc8-427c-83e0-1dde521dbd03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"
Sep 05 06:59:40.332572 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.332548    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbz9g\" (UniqueName: \"kubernetes.io/projected/47f15b0c-ffc8-427c-83e0-1dde521dbd03-kube-api-access-rbz9g\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4ncr7\" (UID: \"47f15b0c-ffc8-427c-83e0-1dde521dbd03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"
Sep 05 06:59:40.365237 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.365214    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"
Sep 05 06:59:40.650450 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:40.650423    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"]
Sep 05 06:59:40.652931 ip-10-0-128-174 kubenswrapper[2573]: W0905 06:59:40.652903    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f15b0c_ffc8_427c_83e0_1dde521dbd03.slice/crio-3fcdc8989d6ac1c7cf357aec8e48f89255bbe1004ce86029f459d6ca0ebd79d6 WatchSource:0}: Error finding container 3fcdc8989d6ac1c7cf357aec8e48f89255bbe1004ce86029f459d6ca0ebd79d6: Status 404 returned error can't find the container with id 3fcdc8989d6ac1c7cf357aec8e48f89255bbe1004ce86029f459d6ca0ebd79d6
Sep 05 06:59:41.244542 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:41.244508    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7" event={"ID":"47f15b0c-ffc8-427c-83e0-1dde521dbd03","Type":"ContainerStarted","Data":"1c18ca563d0ce98156756603778c0a29c92f40eb6046ac800c06d7f6875797b8"}
Sep 05 06:59:41.244969 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:41.244562    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7" event={"ID":"47f15b0c-ffc8-427c-83e0-1dde521dbd03","Type":"ContainerStarted","Data":"3fcdc8989d6ac1c7cf357aec8e48f89255bbe1004ce86029f459d6ca0ebd79d6"}
Sep 05 06:59:41.244969 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:41.244617    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"
Sep 05 06:59:41.271585 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:41.271536    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7" podStartSLOduration=1.271519882 podStartE2EDuration="1.271519882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:59:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:59:41.269317501 +0000 UTC m=+2029.637364711" watchObservedRunningTime="2026-09-05 06:59:41.271519882 +0000 UTC m=+2029.639567092"
Sep 05 06:59:52.251481 ip-10-0-128-174 kubenswrapper[2573]: I0905 06:59:52.251453    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4ncr7"
Sep 05 07:00:27.210440 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:00:27.210369    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 07:00:40.211825 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:00:40.211792    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 07:00:52.259356 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:00:52.259323    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 07:00:52.265894 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:00:52.265872    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 07:00:52.427742 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:00:52.427697    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 07:00:53.716026 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:00:53.715975    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 07:01:10.005763 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.005728    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"]
Sep 05 07:01:10.009470 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.009453    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.020045 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.020020    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"]
Sep 05 07:01:10.141074 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.141036    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-openshift-service-ca\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.141074 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.141075    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd94n\" (UniqueName: \"kubernetes.io/projected/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-kube-api-access-gd94n\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.141362 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.141097    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-maas-api-tls\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.141362 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.141207    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-metrics-certs\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.242416 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.242383    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-openshift-service-ca\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.242416 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.242420    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd94n\" (UniqueName: \"kubernetes.io/projected/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-kube-api-access-gd94n\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.242680 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.242458    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-maas-api-tls\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.242680 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.242484    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-metrics-certs\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.243191 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.243162    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-openshift-service-ca\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.245019 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.244994    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-maas-api-tls\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.245315 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.245297    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-metrics-certs\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.253497 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.253472    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd94n\" (UniqueName: \"kubernetes.io/projected/778b0f4a-e562-4dd5-ab4d-7c7cb4843f85-kube-api-access-gd94n\") pod \"maas-api-c6866d55f-c6nqt\" (UID: \"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85\") " pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.322970 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.322902    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:10.600804 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.600654    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"]
Sep 05 07:01:10.603543 ip-10-0-128-174 kubenswrapper[2573]: W0905 07:01:10.603515    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod778b0f4a_e562_4dd5_ab4d_7c7cb4843f85.slice/crio-e996bc20ff17ff227711e9ff5e78310807c47ad87d5007b31abf44e48fb59ab8 WatchSource:0}: Error finding container e996bc20ff17ff227711e9ff5e78310807c47ad87d5007b31abf44e48fb59ab8: Status 404 returned error can't find the container with id e996bc20ff17ff227711e9ff5e78310807c47ad87d5007b31abf44e48fb59ab8
Sep 05 07:01:10.662030 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:10.661992    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt" event={"ID":"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85","Type":"ContainerStarted","Data":"e996bc20ff17ff227711e9ff5e78310807c47ad87d5007b31abf44e48fb59ab8"}
Sep 05 07:01:11.667643 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:11.667612    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt" event={"ID":"778b0f4a-e562-4dd5-ab4d-7c7cb4843f85","Type":"ContainerStarted","Data":"e082857bdb4bc292697512d7aacf3272a6b3b876a7eafa4f06e3ec0ee899e758"}
Sep 05 07:01:11.668020 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:11.667704    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:11.686882 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:11.686825    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt" podStartSLOduration=2.298522017 podStartE2EDuration="2.686809285s" totalImagesPullingTime="388.287268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:01:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:01:11.684967762 +0000 UTC m=+2120.053014972" watchObservedRunningTime="2026-09-05 07:01:11.686809285 +0000 UTC m=+2120.054856549"
Sep 05 07:01:17.678034 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:17.678003    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-c6866d55f-c6nqt"
Sep 05 07:01:25.409080 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:25.409046    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"]
Sep 05 07:01:25.409572 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:25.409259    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8" podUID="8339082b-fd9e-41c6-b632-900dd2093133" containerName="maas-api" containerID="cri-o://f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461" gracePeriod=30
Sep 05 07:01:26.654984 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.654954    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 07:01:26.689027 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.688999    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-metrics-certs\") pod \"8339082b-fd9e-41c6-b632-900dd2093133\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") "
Sep 05 07:01:26.689197 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.689103    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n8cx\" (UniqueName: \"kubernetes.io/projected/8339082b-fd9e-41c6-b632-900dd2093133-kube-api-access-8n8cx\") pod \"8339082b-fd9e-41c6-b632-900dd2093133\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") "
Sep 05 07:01:26.689197 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.689121    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8339082b-fd9e-41c6-b632-900dd2093133-openshift-service-ca\") pod \"8339082b-fd9e-41c6-b632-900dd2093133\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") "
Sep 05 07:01:26.689197 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.689140    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-maas-api-tls\") pod \"8339082b-fd9e-41c6-b632-900dd2093133\" (UID: \"8339082b-fd9e-41c6-b632-900dd2093133\") "
Sep 05 07:01:26.689572 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.689548    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8339082b-fd9e-41c6-b632-900dd2093133-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8339082b-fd9e-41c6-b632-900dd2093133" (UID: "8339082b-fd9e-41c6-b632-900dd2093133"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 07:01:26.691153 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.691120    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8339082b-fd9e-41c6-b632-900dd2093133" (UID: "8339082b-fd9e-41c6-b632-900dd2093133"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:01:26.691291 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.691268    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8339082b-fd9e-41c6-b632-900dd2093133" (UID: "8339082b-fd9e-41c6-b632-900dd2093133"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:01:26.691540 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.691520    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8339082b-fd9e-41c6-b632-900dd2093133-kube-api-access-8n8cx" (OuterVolumeSpecName: "kube-api-access-8n8cx") pod "8339082b-fd9e-41c6-b632-900dd2093133" (UID: "8339082b-fd9e-41c6-b632-900dd2093133"). InnerVolumeSpecName "kube-api-access-8n8cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:01:26.743141 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.743116    2573 generic.go:356] "Generic (PLEG): container finished" podID="8339082b-fd9e-41c6-b632-900dd2093133" containerID="f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461" exitCode=0
Sep 05 07:01:26.743254 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.743179    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8" event={"ID":"8339082b-fd9e-41c6-b632-900dd2093133","Type":"ContainerDied","Data":"f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461"}
Sep 05 07:01:26.743254 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.743182    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"
Sep 05 07:01:26.743254 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.743220    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8" event={"ID":"8339082b-fd9e-41c6-b632-900dd2093133","Type":"ContainerDied","Data":"00481e499fe49991624a4955434223c83702ae33554713e1296fa9e84e39faec"}
Sep 05 07:01:26.743254 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.743240    2573 scope.go:117] "RemoveContainer" containerID="f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461"
Sep 05 07:01:26.752326 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.752306    2573 scope.go:117] "RemoveContainer" containerID="f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461"
Sep 05 07:01:26.752615 ip-10-0-128-174 kubenswrapper[2573]: E0905 07:01:26.752595    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461\": container with ID starting with f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461 not found: ID does not exist" containerID="f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461"
Sep 05 07:01:26.752696 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.752622    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461"} err="failed to get container status \"f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461\": rpc error: code = NotFound desc = could not find container \"f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461\": container with ID starting with f63ba1aa47f303473c3c0805f7dbbf6b842e04393e395717b8aaa5255aa8a461 not found: ID does not exist"
Sep 05 07:01:26.765409 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.765383    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"]
Sep 05 07:01:26.769670 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.769642    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-8457d65c99-nj7r8"]
Sep 05 07:01:26.790673 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.790649    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n8cx\" (UniqueName: \"kubernetes.io/projected/8339082b-fd9e-41c6-b632-900dd2093133-kube-api-access-8n8cx\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 07:01:26.790673 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.790671    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8339082b-fd9e-41c6-b632-900dd2093133-openshift-service-ca\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 07:01:26.790804 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.790682    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-maas-api-tls\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 07:01:26.790804 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:26.790690    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8339082b-fd9e-41c6-b632-900dd2093133-metrics-certs\") on node \"ip-10-0-128-174.ec2.internal\" DevicePath \"\""
Sep 05 07:01:28.156692 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:28.156660    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8339082b-fd9e-41c6-b632-900dd2093133" path="/var/lib/kubelet/pods/8339082b-fd9e-41c6-b632-900dd2093133/volumes"
Sep 05 07:01:50.603795 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:50.603715    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bcd5999b5-7p5w9_769dff76-6513-4b94-834e-7384a6c0fae5/manager/0.log"
Sep 05 07:01:51.101347 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:51.101318    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555dc54ddd-nmjt2_55738568-3181-4961-a96a-6153a9456780/copy-manifests/0.log"
Sep 05 07:01:51.108737 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:51.108718    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555dc54ddd-nmjt2_55738568-3181-4961-a96a-6153a9456780/manager/0.log"
Sep 05 07:01:51.708274 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:51.708249    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c6866d55f-c6nqt_778b0f4a-e562-4dd5-ab4d-7c7cb4843f85/maas-api/0.log"
Sep 05 07:01:52.040123 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:52.040091    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-zzcxs_83279cb2-f0e0-4d8f-a811-cdcfede4dba6/postgres/0.log"
Sep 05 07:01:53.136812 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:53.136782    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-d8qwd_9e9a0e3c-9095-4760-b88a-b9b364bba2cc/kuadrant-console-plugin/0.log"
Sep 05 07:01:53.257646 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:53.257615    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-4ncr7_47f15b0c-ffc8-427c-83e0-1dde521dbd03/manager/0.log"
Sep 05 07:01:54.211651 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:54.211624    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd96764bd-8zzgm_806212c4-b028-46c6-acc8-0cd1c250c7a8/kube-auth-proxy/0.log"
Sep 05 07:01:54.579871 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:54.579821    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sslnv_9a9386d4-aa77-4949-af89-303484782499/payload-processing/0.log"
Sep 05 07:01:55.371884 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:01:55.371840    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8x8vj_ed79bed1-2aea-40e5-8a55-4bbcaf253c44/main/0.log"
Sep 05 07:02:01.656756 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:01.656720    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 07:02:01.760076 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:01.760047    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 07:02:04.668563 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:04.668532    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cjd64_ac5d72db-04fc-4fbc-9db7-959c7458ad90/global-pull-secret-syncer/0.log"
Sep 05 07:02:04.798968 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:04.798938    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log"
Sep 05 07:02:04.882347 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:04.882319    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-174.ec2.internal_ddc0eaee28eb4599cb47316bccad4632/haproxy/0.log"
Sep 05 07:02:10.086625 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:10.086596    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-d8qwd_9e9a0e3c-9095-4760-b88a-b9b364bba2cc/kuadrant-console-plugin/0.log"
Sep 05 07:02:10.129234 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:10.129201    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-4ncr7_47f15b0c-ffc8-427c-83e0-1dde521dbd03/manager/0.log"
Sep 05 07:02:11.649132 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:11.649105    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0319437e-7daa-42ba-935d-683101d2980e/alertmanager/0.log"
Sep 05 07:02:11.673047 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:11.673022    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0319437e-7daa-42ba-935d-683101d2980e/config-reloader/0.log"
Sep 05 07:02:11.696030 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:11.695967    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0319437e-7daa-42ba-935d-683101d2980e/kube-rbac-proxy-web/0.log"
Sep 05 07:02:11.718578 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:11.718559    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0319437e-7daa-42ba-935d-683101d2980e/kube-rbac-proxy/0.log"
Sep 05 07:02:11.741603 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:11.741584    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0319437e-7daa-42ba-935d-683101d2980e/kube-rbac-proxy-metric/0.log"
Sep 05 07:02:11.770110 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:11.770095    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0319437e-7daa-42ba-935d-683101d2980e/prom-label-proxy/0.log"
Sep 05 07:02:11.800408 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:11.800372    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0319437e-7daa-42ba-935d-683101d2980e/init-config-reloader/0.log"
Sep 05 07:02:11.845534 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:11.845510    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-d97bv_d2cba025-8884-4854-9836-a04f737e6ec1/cluster-monitoring-operator/0.log"
Sep 05 07:02:12.170652 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.170621    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ln2dx_16ede8b1-46ac-4eb5-811a-bf567cd0dbf9/node-exporter/0.log"
Sep 05 07:02:12.193655 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.193629    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ln2dx_16ede8b1-46ac-4eb5-811a-bf567cd0dbf9/kube-rbac-proxy/0.log"
Sep 05 07:02:12.219160 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.219138    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ln2dx_16ede8b1-46ac-4eb5-811a-bf567cd0dbf9/init-textfile/0.log"
Sep 05 07:02:12.430086 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.430056    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd293127-52cd-46ea-aa6d-6c3d8af86732/prometheus/0.log"
Sep 05 07:02:12.451764 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.451737    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd293127-52cd-46ea-aa6d-6c3d8af86732/config-reloader/0.log"
Sep 05 07:02:12.473522 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.473502    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd293127-52cd-46ea-aa6d-6c3d8af86732/thanos-sidecar/0.log"
Sep 05 07:02:12.500789 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.500770    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd293127-52cd-46ea-aa6d-6c3d8af86732/kube-rbac-proxy-web/0.log"
Sep 05 07:02:12.524164 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.524142    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd293127-52cd-46ea-aa6d-6c3d8af86732/kube-rbac-proxy/0.log"
Sep 05 07:02:12.554312 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.554289    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd293127-52cd-46ea-aa6d-6c3d8af86732/kube-rbac-proxy-thanos/0.log"
Sep 05 07:02:12.578503 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.578446    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd293127-52cd-46ea-aa6d-6c3d8af86732/init-config-reloader/0.log"
Sep 05 07:02:12.610107 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.610086    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-ldfkm_5339d188-3c4e-49bc-8ed0-d1f8eda83ff9/prometheus-operator/0.log"
Sep 05 07:02:12.631526 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.631504    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-ldfkm_5339d188-3c4e-49bc-8ed0-d1f8eda83ff9/kube-rbac-proxy/0.log"
Sep 05 07:02:12.662799 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.662774    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-gv7p6_276610f9-7096-4300-ba3d-35e5088a8b62/prometheus-operator-admission-webhook/0.log"
Sep 05 07:02:12.709383 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.709360    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9bbfdb7c9-4tsjd_65681bbd-95a6-4f05-9231-a98ad19c022f/telemeter-client/0.log"
Sep 05 07:02:12.734727 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.734701    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9bbfdb7c9-4tsjd_65681bbd-95a6-4f05-9231-a98ad19c022f/reload/0.log"
Sep 05 07:02:12.757080 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:12.757054    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9bbfdb7c9-4tsjd_65681bbd-95a6-4f05-9231-a98ad19c022f/kube-rbac-proxy/0.log"
Sep 05 07:02:14.418394 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.418363    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"]
Sep 05 07:02:14.418842 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.418750    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8339082b-fd9e-41c6-b632-900dd2093133" containerName="maas-api"
Sep 05 07:02:14.418842 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.418763    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8339082b-fd9e-41c6-b632-900dd2093133" containerName="maas-api"
Sep 05 07:02:14.418842 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.418821    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8339082b-fd9e-41c6-b632-900dd2093133" containerName="maas-api"
Sep 05 07:02:14.421997 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.421977    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.424498 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.424473    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tj5rw\"/\"kube-root-ca.crt\""
Sep 05 07:02:14.424498 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.424489    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tj5rw\"/\"openshift-service-ca.crt\""
Sep 05 07:02:14.425381 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.425367    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tj5rw\"/\"default-dockercfg-9d82n\""
Sep 05 07:02:14.430403 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.430376    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"]
Sep 05 07:02:14.519465 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.519422    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-proc\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.519639 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.519496    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-sys\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.519639 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.519537    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grshc\" (UniqueName: \"kubernetes.io/projected/6335facd-208e-40a2-9233-2a629bf36a46-kube-api-access-grshc\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.519639 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.519600    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-lib-modules\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.519639 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.519625    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-podres\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.620753 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.620723    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grshc\" (UniqueName: \"kubernetes.io/projected/6335facd-208e-40a2-9233-2a629bf36a46-kube-api-access-grshc\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.620928 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.620763    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-lib-modules\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.620928 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.620784    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-podres\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.620928 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.620835    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-proc\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.620928 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.620894    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-sys\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.621102 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.620925    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-lib-modules\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.621102 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.620944    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-proc\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.621102 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.620980    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-podres\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.621102 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.620985    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6335facd-208e-40a2-9233-2a629bf36a46-sys\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.631219 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.631189    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grshc\" (UniqueName: \"kubernetes.io/projected/6335facd-208e-40a2-9233-2a629bf36a46-kube-api-access-grshc\") pod \"perf-node-gather-daemonset-wpvzg\" (UID: \"6335facd-208e-40a2-9233-2a629bf36a46\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.732486 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.732397    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:14.829819 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.829793    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/2.log"
Sep 05 07:02:14.833478 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:14.833455    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4n587_aa4da948-54e4-40a2-8c68-e6a250c5f2be/console-operator/3.log"
Sep 05 07:02:15.000166 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:15.000140    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"]
Sep 05 07:02:15.002184 ip-10-0-128-174 kubenswrapper[2573]: W0905 07:02:15.002157    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6335facd_208e_40a2_9233_2a629bf36a46.slice/crio-c75e0445aaed1477c687498ef536520c6d8a91951a08021c0ef510a9e42b9eb3 WatchSource:0}: Error finding container c75e0445aaed1477c687498ef536520c6d8a91951a08021c0ef510a9e42b9eb3: Status 404 returned error can't find the container with id c75e0445aaed1477c687498ef536520c6d8a91951a08021c0ef510a9e42b9eb3
Sep 05 07:02:15.417751 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:15.417662    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-rrktx_574cc721-7a78-4807-8f0b-c9268affb086/download-server/0.log"
Sep 05 07:02:15.973570 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:15.973535    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg" event={"ID":"6335facd-208e-40a2-9233-2a629bf36a46","Type":"ContainerStarted","Data":"6a96973a125bdb180adbb92e6281ba5ca81aedd109b8d5fac9a0a51f9074bb9a"}
Sep 05 07:02:15.974050 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:15.973594    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg" event={"ID":"6335facd-208e-40a2-9233-2a629bf36a46","Type":"ContainerStarted","Data":"c75e0445aaed1477c687498ef536520c6d8a91951a08021c0ef510a9e42b9eb3"}
Sep 05 07:02:15.974050 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:15.973623    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:15.991243 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:15.991197    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg" podStartSLOduration=1.991181031 podStartE2EDuration="1.991181031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:02:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:02:15.990502171 +0000 UTC m=+2184.358549384" watchObservedRunningTime="2026-09-05 07:02:15.991181031 +0000 UTC m=+2184.359228241"
Sep 05 07:02:17.249710 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:17.249682    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xpm9h_ae33df4f-832d-4ee1-83b7-798f60f540d8/dns/0.log"
Sep 05 07:02:17.272505 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:17.272477    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xpm9h_ae33df4f-832d-4ee1-83b7-798f60f540d8/kube-rbac-proxy/0.log"
Sep 05 07:02:17.323940 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:17.323910    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nq8t2_e7095477-5141-481a-8b27-64e4c36845cb/dns-node-resolver/0.log"
Sep 05 07:02:17.912624 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:17.912592    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7pdxl_4bad5cfd-a364-42c1-90dd-e2f7ddc39e64/node-ca/0.log"
Sep 05 07:02:19.241720 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:19.241688    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd96764bd-8zzgm_806212c4-b028-46c6-acc8-0cd1c250c7a8/kube-auth-proxy/0.log"
Sep 05 07:02:19.346729 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:19.346699    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sslnv_9a9386d4-aa77-4949-af89-303484782499/payload-processing/0.log"
Sep 05 07:02:20.287265 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:20.287235    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-llbmp_5d2c676f-9881-4180-9aa4-2cc3757adfa0/serve-healthcheck-canary/0.log"
Sep 05 07:02:20.928762 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:20.928738    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-npbp9_6df16279-9467-4374-b506-15aeed9823c0/kube-rbac-proxy/0.log"
Sep 05 07:02:20.952562 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:20.952540    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-npbp9_6df16279-9467-4374-b506-15aeed9823c0/exporter/0.log"
Sep 05 07:02:20.977261 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:20.977238    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-npbp9_6df16279-9467-4374-b506-15aeed9823c0/extractor/0.log"
Sep 05 07:02:21.988878 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:21.988841    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-wpvzg"
Sep 05 07:02:23.483460 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:23.483429    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bcd5999b5-7p5w9_769dff76-6513-4b94-834e-7384a6c0fae5/manager/0.log"
Sep 05 07:02:23.662666 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:23.662599    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555dc54ddd-nmjt2_55738568-3181-4961-a96a-6153a9456780/manager/0.log"
Sep 05 07:02:23.688531 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:23.688505    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555dc54ddd-nmjt2_55738568-3181-4961-a96a-6153a9456780/copy-manifests/0.log"
Sep 05 07:02:25.164931 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:25.164906    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66f494d4f4-hlr98_35c3ef1a-ae1e-4fcd-a407-e60cb4c67fa0/manager/0.log"
Sep 05 07:02:25.268834 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:25.268808    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd96764bd-8zzgm_806212c4-b028-46c6-acc8-0cd1c250c7a8/kube-auth-proxy/0.log"
Sep 05 07:02:25.358247 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:25.358219    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sslnv_9a9386d4-aa77-4949-af89-303484782499/payload-processing/0.log"
Sep 05 07:02:26.882119 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:26.882083    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024
Sep 05 07:02:27.138815 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:27.138737    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd96764bd-8zzgm_806212c4-b028-46c6-acc8-0cd1c250c7a8/kube-auth-proxy/0.log"
Sep 05 07:02:27.221170 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:27.221142    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-sslnv_9a9386d4-aa77-4949-af89-303484782499/payload-processing/0.log"
Sep 05 07:02:29.057934 ip-10-0-128-174 kubenswrapper[2573]: I0905 07:02:29.057902    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xv7j_8cae7568-294a-4c0e-adf9-d9581668c54e/konnectivity-agent/0.log" limit=1024