Sep 08 14:32:16.083607 ip-10-0-137-6 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 08 14:32:16.083617 ip-10-0-137-6 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 08 14:32:16.083624 ip-10-0-137-6 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 08 14:32:16.083772 ip-10-0-137-6 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 08 14:32:26.172085 ip-10-0-137-6 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 08 14:32:26.172105 ip-10-0-137-6 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 734d1f64787f4326aecc5f2cc3afec46 --
Sep 08 14:34:09.142133 ip-10-0-137-6 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 14:34:09.562762 ip-10-0-137-6 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 14:34:09.562762 ip-10-0-137-6 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 14:34:09.562762 ip-10-0-137-6 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 14:34:09.562762 ip-10-0-137-6 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 14:34:09.562762 ip-10-0-137-6 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 14:34:09.564952 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.564873    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 14:34:09.570117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570102    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:34:09.570117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570117    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570121    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570124    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570127    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570130    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570133    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570136    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570141    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570144    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570147    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570150    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570152    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570155    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570157    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570160    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570162    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570165    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570168    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570174    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:34:09.570183 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570176    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570179    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570182    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570184    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570187    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570191    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570195    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570199    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570202    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570205    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570208    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570210    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570213    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570215    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570218    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570220    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570223    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570226    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570228    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:34:09.570633 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570231    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570234    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570236    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570239    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570241    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570244    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570246    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570249    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570251    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570253    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570256    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570258    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570260    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570263    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570265    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570268    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570271    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570274    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570277    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570279    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:34:09.571109 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570282    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570284    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570287    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570289    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570292    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570294    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570297    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570299    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570302    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570304    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570307    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570309    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570312    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570314    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570317    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570319    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570322    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570324    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570328    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570330    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:34:09.571600 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570333    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570335    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570338    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570340    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570343    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570345    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570347    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570351    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570354    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570356    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570359    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570361    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570365    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570369    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570371    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570374    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570376    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570378    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570381    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:34:09.572120 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.570383    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571094    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571102    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571106    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571109    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571112    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571116    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571118    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571121    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571124    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571126    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571129    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571131    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571134    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571137    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571139    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571142    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571145    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571147    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571150    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:34:09.572590 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571153    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571155    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571158    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571161    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571164    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571166    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571169    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571172    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571174    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571176    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571179    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571181    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571184    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571186    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571189    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571191    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571195    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571198    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571200    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571203    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:34:09.573099 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571206    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571208    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571212    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571215    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571218    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571223    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571226    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571228    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571231    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571233    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571235    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571238    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571242    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571244    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571247    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571250    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571252    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571255    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571257    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:34:09.573584 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571260    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571262    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571265    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571267    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571270    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571272    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571275    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571277    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571280    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571282    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571286    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571304    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571307    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571311    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571313    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571316    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571319    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571322    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571326    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:34:09.574063 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571328    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571331    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571333    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571336    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571339    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571341    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571344    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571346    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571349    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571352    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571354    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571357    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571359    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571362    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571364    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571367    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571369    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571372    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571374    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571377    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:34:09.574625 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571380    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571382    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571742    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571747    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571750    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571753    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571756    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571759    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571762    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571765    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571767    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571771    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571774    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571777    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571779    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571781    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571784    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571787    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571790    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571792    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:34:09.575149 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571795    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571797    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571800    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571802    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571805    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571809    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571813    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571816    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571818    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571821    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571825    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571843    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571845    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571848    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571850    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571853    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571856    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571858    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571861    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:34:09.575648 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571864    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571867    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571869    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571872    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571876    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571878    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571881    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571883    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571886    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571888    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571890    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571893    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571896    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571898    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571901    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571903    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571906    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571908    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571911    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:34:09.576118 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571914    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571916    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571918    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571921    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571924    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571926    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571929    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571931    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571933    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571936    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571938    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571941    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571944    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571946    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571949    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571952    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571954    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571958    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571961    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571963    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571969    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:34:09.576603 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571971    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571974    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571976    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571979    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571982    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571984    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571987    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571989    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571992    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571994    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571997    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.571999    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572002    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572004    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572007    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572009    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572012    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572014    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572016    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572019    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:34:09.577155 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572022    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572025    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572095    2583 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572127    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572133    2583 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572138    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572142    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572145    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572150    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572155    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572158    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572161    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572165    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572168    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572170    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572173    2583 flags.go:64] FLAG: --cgroup-root=""
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572176    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572179    2583 flags.go:64] FLAG: --client-ca-file=""
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572182    2583 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572185    2583 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572189    2583 flags.go:64] FLAG: --cluster-domain=""
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572192    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572194    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 14:34:09.577635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572197    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572201    2583 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572204    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572207    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572210    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572213    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572216    2583 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572218    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572221    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572224    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572227    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572230    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572233    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572236    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572239    2583 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572241    2583 flags.go:64] FLAG: --enable-server="true"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572245    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572249    2583 flags.go:64] FLAG: --event-burst="100"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572252    2583 flags.go:64] FLAG: --event-qps="50"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572255    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572258    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572262    2583 flags.go:64] FLAG: --eviction-hard=""
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572265    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572268    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572271    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 14:34:09.578244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572274    2583 flags.go:64] FLAG: --eviction-soft=""
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572277    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572280    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572282    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572285    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572288    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572291    2583 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572294    2583 flags.go:64] FLAG: --feature-gates=""
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572297    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572300    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572303    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572306    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572309    2583 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572312    2583 flags.go:64] FLAG: --help="false"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572315    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-137-6.ec2.internal"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572318    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572321    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572324    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572327    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572330    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572333    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572336    2583 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572338    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 14:34:09.578869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572341    2583 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572345    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572348    2583 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572351    2583 flags.go:64] FLAG: --kube-reserved=""
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572354    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572357    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572360    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572362    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572365    2583 flags.go:64] FLAG: --lock-file=""
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572368    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572371    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572374    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572384    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572387    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572390    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572392    2583 flags.go:64] FLAG: --logging-format="text"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572395    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572398    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572401    2583 flags.go:64] FLAG: --manifest-url=""
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572404    2583 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572408    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572411    2583 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572415    2583 flags.go:64] FLAG: --max-pods="110"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572418    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572421    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 14:34:09.579478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572424    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572427    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572430    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572432    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572436    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572443    2583 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572445    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572448    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572451    2583 flags.go:64] FLAG: --pod-cidr=""
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572455    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572459    2583 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572462    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572465    2583 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572468    2583 flags.go:64] FLAG: --port="10250"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572471    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572473    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e898f8c5ba93db9d"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572476    2583 flags.go:64] FLAG: --qos-reserved=""
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572479    2583 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572482    2583 flags.go:64] FLAG: --register-node="true"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572485    2583 flags.go:64] FLAG: --register-with-taints=""
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572488    2583 flags.go:64] FLAG: --registry-burst="10"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572491    2583 flags.go:64] FLAG: --registry-qps="5"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572494    2583 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572496    2583 flags.go:64] FLAG: --reserved-memory=""
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572500    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 14:34:09.580088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572503    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572505    2583 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572508    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572511    2583 flags.go:64] FLAG: --runonce="false"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572514    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572516    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572519    2583 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572522    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572525    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572528    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572531    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572534    2583 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572536    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572539    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572543    2583 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572546    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572549    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572552    2583 flags.go:64] FLAG: --system-cgroups=""
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572554    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572561    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572563    2583 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572567    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572571    2583 flags.go:64] FLAG: --tls-min-version=""
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572573    2583 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572576    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 14:34:09.580711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572579    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572582    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572585    2583 flags.go:64] FLAG: --v="2"
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572588    2583 flags.go:64] FLAG: --version="false"
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572593    2583 flags.go:64] FLAG: --vmodule=""
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572597    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.572600    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572726    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572731    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572735    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572739    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572741    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572744    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572746    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572749    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572752    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572754    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572757    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572759    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572762    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572764    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:34:09.581322 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572768    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572770    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572773    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572775    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572778    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572782    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572784    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572787    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572789    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572793    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572796    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572799    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572802    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572805    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572807    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572810    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572813    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572816    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572819    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:34:09.581898 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572821    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572824    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572840    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572843    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572845    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572848    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572851    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572854    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572858    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572861    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572864    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572867    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572869    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572873    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572876    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572879    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572881    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572884    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572888    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:34:09.582362 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572891    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572893    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572896    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572898    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572901    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572903    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572906    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572908    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572911    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572913    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572916    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572918    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572921    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572923    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572926    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572928    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572931    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572933    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572936    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572939    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:34:09.582812 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572941    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572943    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572946    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572948    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572951    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572953    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572958    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572961    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572963    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572966    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572968    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572972    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572975    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572978    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572980    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572983    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572985    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572988    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572990    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572993    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:34:09.583311 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572995    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.572997    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.573000    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.573002    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.573005    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.573008    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.573010    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.573601    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.580396    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.580411    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.581190    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.581197    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581255    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581260    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581264    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:34:09.583802 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581267    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581270    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581273    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581276    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581279    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581283    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581286    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581288    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581291    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581295    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581298    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581300    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581303    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581305    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581308    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581310    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581313    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581315    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581318    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581321    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:34:09.584184 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581323    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581326    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581331    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581334    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581337    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581340    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581344    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581347    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581350    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581352    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581354    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581357    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581360    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581363    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581367    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581369    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581372    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581374    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581377    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:34:09.584679 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581379    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581382    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581384    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581388    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581391    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581393    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581396    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581398    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581401    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581403    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581406    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581409    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581411    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581414    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581416    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581420    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581423    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581426    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581429    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581432    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:34:09.585235 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581435    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581438    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581440    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581443    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581445    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581448    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581451    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581453    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581456    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581459    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581462    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581464    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581467    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581469    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581471    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581474    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581477    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581480    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581482    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581485    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:34:09.585730 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581487    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581490    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581492    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581495    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581497    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581500    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581502    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581504    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581507    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581510    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581513    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581515    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581518    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581521    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581523    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581525    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:34:09.586461 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581528    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.581533    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581640    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581645    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581647    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581650    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581654    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581657    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581660    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581663    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581666    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581668    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581671    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581674    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 14:34:09.587125 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581677    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581679    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581682    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581684    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581687    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581690    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581693    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581695    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581698    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581700    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581703    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581705    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581708    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581711    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581713    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581716    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581718    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581721    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581723    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581726    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 14:34:09.587494 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581728    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581731    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581734    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581736    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581738    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581741    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581743    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581746    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581748    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581750    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581753    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581755    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581758    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581760    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581763    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581765    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581767    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581770    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581772    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581774    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 14:34:09.588016 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581777    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581779    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581791    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581794    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581796    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581799    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581803    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581805    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581809    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581812    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581815    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581818    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581821    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581824    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581826    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581846    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581849    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581852    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 14:34:09.588604 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581855    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581859    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581862    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581865    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581867    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581870    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581872    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581876    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581878    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581881    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581883    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581886    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581889    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581891    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581894    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581896    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581899    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581902    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581904    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581907    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 14:34:09.589117 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581910    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581912    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581915    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581917    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581920    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581923    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581925    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581928    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:09.581930    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.581935    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.582049    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.584150    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.585898    2583 server.go:1019] "Starting client certificate rotation"
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.586023    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 14:34:09.589639 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.586094    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 14:34:09.590207 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.588516    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 14:34:09.590207 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.589996    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 14:34:09.607293 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.607265    2583 log.go:25] "Validated CRI v1 runtime API"
Sep 08 14:34:09.610235 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.610116    2583 log.go:25] "Validated CRI v1 image API"
Sep 08 14:34:09.610992 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.610976    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 14:34:09.615274 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.615254    2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9ffe5b90-b26e-456d-8469-5bf93dff57b8:/dev/nvme0n1p3 ba1910e1-232e-4299-909c-2f9706313a56:/dev/nvme0n1p4]
Sep 08 14:34:09.615344 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.615273    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 14:34:09.618919 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.618901    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 14:34:09.621014 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.620891    2583 manager.go:216] Machine: {Timestamp:2026-09-08 14:34:09.619598678 +0000 UTC m=+0.347415683 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103705 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23b0a33338b78580f4f7d2bcc298dd SystemUUID:ec23b0a3-3338-b785-80f4-f7d2bcc298dd BootID:734d1f64-787f-4326-aecc-5f2cc3afec46 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:61:dd:82:b0:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:61:dd:82:b0:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:a4:e8:a3:84:ac Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 14:34:09.621014 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.621005    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 14:34:09.621130 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.621119    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 14:34:09.623996 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.623966    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 14:34:09.624156 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.624000    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-6.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 14:34:09.624905 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.624893    2583 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 14:34:09.624967 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.624912    2583 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 14:34:09.624967 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.624923    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 14:34:09.625568 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.625556    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 14:34:09.626624 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.626612    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 14:34:09.629146 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.629133    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 14:34:09.629281 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.629270    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 14:34:09.629347 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.629310    2583 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 14:34:09.629347 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.629325    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 14:34:09.629347 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.629339    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 14:34:09.629467 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.629352    2583 kubelet.go:398] "Adding apiserver pod source"
Sep 08 14:34:09.629467 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.629364    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 14:34:09.629754 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.629740    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 14:34:09.629853 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.629764    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 14:34:09.632032 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.632012    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 14:34:09.634088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.634073    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 14:34:09.634154 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.634103    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 14:34:09.634257 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.634248    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 14:34:09.634301 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.634262    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 14:34:09.634301 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.634268    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 14:34:09.634301 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.634274    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 14:34:09.634301 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.634282    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 14:34:09.634301 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.634287    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 14:34:09.635040 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.635031    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 14:34:09.635040 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.635040    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 14:34:09.635112 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.635048    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 14:34:09.635112 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.635053    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 14:34:09.635112 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.635062    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 14:34:09.635112 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.635069    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 14:34:09.635912 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.635900    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 14:34:09.635912 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.635910    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 14:34:09.639384 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.639369    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-6.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 14:34:09.639384 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.639377    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 14:34:09.639587 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.639573    2583 server.go:1268] "Started kubelet"
Sep 08 14:34:09.639666 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.639590    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 14:34:09.639666 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.639647    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 14:34:09.639754 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.639686    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 14:34:09.643195 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.643168    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 14:34:09.643743 ip-10-0-137-6 systemd[1]: Started Kubernetes Kubelet.
Sep 08 14:34:09.644592 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.644354    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lwmh6"
Sep 08 14:34:09.645410 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.645376    2583 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 14:34:09.645667 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.645647    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 14:34:09.648194 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.648174    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 14:34:09.649357 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.648347    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-6.ec2.internal.18d35f1bb7424e46  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-6.ec2.internal,UID:ip-10-0-137-6.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-6.ec2.internal,},FirstTimestamp:2026-09-08 14:34:09.639550534 +0000 UTC m=+0.367367541,LastTimestamp:2026-09-08 14:34:09.639550534 +0000 UTC m=+0.367367541,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-6.ec2.internal,}"
Sep 08 14:34:09.649609 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.649592    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 14:34:09.650204 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.650190    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 14:34:09.650311 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.650297    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:09.650734 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.650719    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 14:34:09.650734 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.650728    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 14:34:09.650876 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.650735    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 14:34:09.650876 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.650869    2583 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 14:34:09.650876 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.650877    2583 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 14:34:09.651022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.651012    2583 factory.go:55] Registering systemd factory
Sep 08 14:34:09.651069 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.651028    2583 factory.go:223] Registration of the systemd container factory successfully
Sep 08 14:34:09.651232 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.651218    2583 factory.go:153] Registering CRI-O factory
Sep 08 14:34:09.651281 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.651237    2583 factory.go:223] Registration of the crio container factory successfully
Sep 08 14:34:09.651310 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.651296    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 14:34:09.651351 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.651310    2583 factory.go:103] Registering Raw factory
Sep 08 14:34:09.651351 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.651321    2583 manager.go:1196] Started watching for new ooms in manager
Sep 08 14:34:09.651698 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.651686    2583 manager.go:349] Starting recovery of all containers
Sep 08 14:34:09.653041 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.653024    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lwmh6"
Sep 08 14:34:09.662711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.662696    2583 manager.go:354] Recovery completed
Sep 08 14:34:09.664164 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.664146    2583 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 08 14:34:09.664766 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.664750    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:34:09.667305 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.667293    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:34:09.667903 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.667888    2583 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-6.ec2.internal\" not found" node="ip-10-0-137-6.ec2.internal"
Sep 08 14:34:09.669341 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.669325    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:34:09.669424 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.669353    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:34:09.669424 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.669363    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:34:09.669870 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.669860    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 14:34:09.669916 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.669870    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 14:34:09.669916 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.669874    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:34:09.669916 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.669885    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 14:34:09.670531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.670518    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:34:09.670584 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.670544    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:34:09.670584 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.670558    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:34:09.670996 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.670985    2583 policy_none.go:49] "None policy: Start"
Sep 08 14:34:09.671038 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.671000    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 14:34:09.671038 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.671010    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 14:34:09.671964 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.671954    2583 policy_none.go:47] "Start"
Sep 08 14:34:09.706000 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.705924    2583 manager.go:337] "Starting Device Plugin manager"
Sep 08 14:34:09.706000 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.705960    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 14:34:09.706000 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.705973    2583 server.go:87] "Starting device plugin registration server"
Sep 08 14:34:09.727506 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.706432    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 14:34:09.727506 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.706445    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 14:34:09.727506 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.706585    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 14:34:09.727506 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.706655    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 14:34:09.727506 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.706664    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 14:34:09.727506 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.707132    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 14:34:09.727506 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.707187    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:09.785310 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.785277    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 14:34:09.786612 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.786586    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 14:34:09.786715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.786646    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 14:34:09.786715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.786671    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 14:34:09.786816 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.786715    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 08 14:34:09.789614 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.789599    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:34:09.807241 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.807211    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:34:09.809989 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.809974    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:34:09.810080 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.810007    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:34:09.810080 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.810021    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:34:09.810080 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.810052    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-6.ec2.internal"
Sep 08 14:34:09.819548 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.819503    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-6.ec2.internal"
Sep 08 14:34:09.819548 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.819524    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-6.ec2.internal\": node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:09.836859 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.836820    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:09.887531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.887485    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-6.ec2.internal"]
Sep 08 14:34:09.887631 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.887584    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:34:09.889413 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.889398    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:34:09.889519 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.889430    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:34:09.889519 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.889446    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:34:09.890905 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.890891    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:34:09.891022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.891005    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:09.891074 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.891039    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:34:09.891623 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.891605    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:34:09.891702 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.891636    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:34:09.891702 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.891647    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:34:09.891702 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.891605    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:34:09.891810 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.891716    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:34:09.891810 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.891728    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:34:09.893282 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.893269    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:09.893332 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.893291    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 14:34:09.893924 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.893910    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 14:34:09.894000 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.893934    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 14:34:09.894000 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.893949    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeHasSufficientPID"
Sep 08 14:34:09.910365 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.910346    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-6.ec2.internal\" not found" node="ip-10-0-137-6.ec2.internal"
Sep 08 14:34:09.914755 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.914740    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-6.ec2.internal\" not found" node="ip-10-0-137-6.ec2.internal"
Sep 08 14:34:09.937431 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:09.937413    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:09.954274 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.954256    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15f573fb3bfde2f504806a4ebee87ad0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal\" (UID: \"15f573fb3bfde2f504806a4ebee87ad0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:09.954361 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.954280    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15f573fb3bfde2f504806a4ebee87ad0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal\" (UID: \"15f573fb3bfde2f504806a4ebee87ad0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:09.954361 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:09.954296    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/358c2d635fdf18050ee800796677d30a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-6.ec2.internal\" (UID: \"358c2d635fdf18050ee800796677d30a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:10.037896 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:10.037868    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:10.054528 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.054503    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15f573fb3bfde2f504806a4ebee87ad0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal\" (UID: \"15f573fb3bfde2f504806a4ebee87ad0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:10.054685 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.054534    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15f573fb3bfde2f504806a4ebee87ad0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal\" (UID: \"15f573fb3bfde2f504806a4ebee87ad0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:10.054685 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.054559    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/358c2d635fdf18050ee800796677d30a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-6.ec2.internal\" (UID: \"358c2d635fdf18050ee800796677d30a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:10.054685 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.054612    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/358c2d635fdf18050ee800796677d30a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-6.ec2.internal\" (UID: \"358c2d635fdf18050ee800796677d30a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:10.054685 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.054618    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15f573fb3bfde2f504806a4ebee87ad0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal\" (UID: \"15f573fb3bfde2f504806a4ebee87ad0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:10.054685 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.054650    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15f573fb3bfde2f504806a4ebee87ad0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal\" (UID: \"15f573fb3bfde2f504806a4ebee87ad0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:10.138275 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:10.138196    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:10.212616 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.212574    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:10.217553 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.217536    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:10.239194 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:10.239163    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:10.339652 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:10.339617    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:10.440124 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:10.440067    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:10.540543 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:10.540520    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:10.556906 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.556885    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:34:10.585953 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.585935    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 14:34:10.586328 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.586047    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 14:34:10.586328 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.586084    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 14:34:10.586328 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.586096    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 14:34:10.640584 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:10.640566    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:10.655219 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.655190    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 14:29:09 +0000 UTC" deadline="2028-04-27 00:55:14.489518873 +0000 UTC"
Sep 08 14:34:10.655219 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.655215    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14314h21m3.834308401s"
Sep 08 14:34:10.741682 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:10.741651    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-6.ec2.internal\" not found"
Sep 08 14:34:10.770581 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.770552    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:34:10.850934 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.850909    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:10.864282 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.864260    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 14:34:10.865479 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.865464    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-6.ec2.internal"
Sep 08 14:34:10.875072 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:10.875050    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 14:34:10.998495 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:10.998326    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f573fb3bfde2f504806a4ebee87ad0.slice/crio-c826b1d33830a404a6b2ad1ae992b86a2b85cc07b6f0100407f751a8ca025e51 WatchSource:0}: Error finding container c826b1d33830a404a6b2ad1ae992b86a2b85cc07b6f0100407f751a8ca025e51: Status 404 returned error can't find the container with id c826b1d33830a404a6b2ad1ae992b86a2b85cc07b6f0100407f751a8ca025e51
Sep 08 14:34:11.002704 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.002691    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:34:11.520728 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.520701    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:34:11.582293 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.582268    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:34:11.630508 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.630489    2583 apiserver.go:52] "Watching apiserver"
Sep 08 14:34:11.638754 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.638729    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 14:34:11.639057 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.639038    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-ntzrt","kube-system/kube-apiserver-proxy-ip-10-0-137-6.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2","openshift-cluster-node-tuning-operator/tuned-qljrl","openshift-dns/node-resolver-vsxtr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal","openshift-multus/multus-additional-cni-plugins-5wbz8","openshift-ovn-kubernetes/ovnkube-node-9lt7h","kube-system/konnectivity-agent-fjchd","openshift-image-registry/node-ca-hvzkl","openshift-multus/multus-dn9wg","openshift-multus/network-metrics-daemon-tmqvh","openshift-network-diagnostics/network-check-target-php6q","openshift-network-operator/iptables-alerter-xpklc"]
Sep 08 14:34:11.641030 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.641017    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.642023 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.642002    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:11.642107 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:11.642064    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:11.643051 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.643036    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.643753 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.643729    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 14:34:11.643807 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.643758    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zwmt4\""
Sep 08 14:34:11.643992 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.643966    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.643992 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.643971    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 14:34:11.644104 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.643997    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 14:34:11.644104 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.644047    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 14:34:11.644104 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.644047    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 14:34:11.645174 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.645157    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vsxtr"
Sep 08 14:34:11.645448 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.645433    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 14:34:11.645527 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.645452    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 14:34:11.645527 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.645436    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 14:34:11.645854 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.645839    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bmhmz\""
Sep 08 14:34:11.646189 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.646172    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xpklc"
Sep 08 14:34:11.646482 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.646465    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p84wr\""
Sep 08 14:34:11.647192 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.647177    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:11.647530 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.647500    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 14:34:11.647602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.647556    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 14:34:11.647665 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.647623    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 14:34:11.647665 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.647649    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 14:34:11.647745 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.647691    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r4d8f\""
Sep 08 14:34:11.648414 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.648326    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:11.648414 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:11.648389    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:11.648792 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.648774    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 14:34:11.648888 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.648788    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 14:34:11.648888 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.648789    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-985fb\""
Sep 08 14:34:11.649022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.649005    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 14:34:11.649590 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.649575    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.650001 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.649957    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 14:34:11.650567 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.650552    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 14:34:11.650638 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.650618    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9s9wc\""
Sep 08 14:34:11.650683 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.650618    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 14:34:11.651059 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.651044    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hvzkl"
Sep 08 14:34:11.651911 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.651893    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 14:34:11.651997 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.651913    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fhgpg\""
Sep 08 14:34:11.651997 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.651922    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 14:34:11.652447 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.652393    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.653623 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.653591    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 14:34:11.653966 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.653946    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 14:34:11.654035 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.653998    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 14:34:11.655012 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.654996    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 14:34:11.655106 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.655095    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 14:34:11.655162 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.655115    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:11.655243 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:11.655226    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:11.655810 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.655791    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7glpg\""
Sep 08 14:34:11.655959 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.655944    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 14:34:11.656169 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.656152    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 14:34:11.656685 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.656669    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wbdgd\""
Sep 08 14:34:11.656766 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.656709    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 14:34:11.663909 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.663891    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8grbd\" (UniqueName: \"kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd\") pod \"network-check-target-php6q\" (UID: \"ac6d150a-3459-4e57-93c5-2c52ec02c6a3\") " pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:11.663993 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.663928    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-var-lib-kubelet\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.663993 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.663957    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-device-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.663993 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.663987    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:11.664138 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664018    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-systemd-units\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.664138 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664046    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-run-openvswitch\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.664138 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664074    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55f744b4-526c-4826-8bc5-46ace246b9d7-env-overrides\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.664138 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664121    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a15e5b5-3ba1-4587-943a-a4989e67b57b-host\") pod \"node-ca-hvzkl\" (UID: \"2a15e5b5-3ba1-4587-943a-a4989e67b57b\") " pod="openshift-image-registry/node-ca-hvzkl"
Sep 08 14:34:11.664296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664170    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-run\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.664296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664204    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d88ea008-9ce7-4a83-abd8-bbde485345a3-tmp-dir\") pod \"node-resolver-vsxtr\" (UID: \"d88ea008-9ce7-4a83-abd8-bbde485345a3\") " pod="openshift-dns/node-resolver-vsxtr"
Sep 08 14:34:11.664296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664230    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kpsj\" (UniqueName: \"kubernetes.io/projected/d88ea008-9ce7-4a83-abd8-bbde485345a3-kube-api-access-9kpsj\") pod \"node-resolver-vsxtr\" (UID: \"d88ea008-9ce7-4a83-abd8-bbde485345a3\") " pod="openshift-dns/node-resolver-vsxtr"
Sep 08 14:34:11.664296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664244    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78adf8d2-113a-4251-a33f-66236876be03-cnibin\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.664296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664257    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-run-netns\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.664296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664270    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-kubernetes\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.664495 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664313    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3ed718d-5c96-403d-85c2-6e1cbca41b58-iptables-alerter-script\") pod \"iptables-alerter-xpklc\" (UID: \"b3ed718d-5c96-403d-85c2-6e1cbca41b58\") " pod="openshift-network-operator/iptables-alerter-xpklc"
Sep 08 14:34:11.664495 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664359    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78adf8d2-113a-4251-a33f-66236876be03-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.664495 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664389    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-node-log\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.664495 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664419    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-cni-bin\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.664495 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664456    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-cni-netd\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.664495 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664476    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-host\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.664726 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664500    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-etc-selinux\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.664726 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664520    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3ed718d-5c96-403d-85c2-6e1cbca41b58-host-slash\") pod \"iptables-alerter-xpklc\" (UID: \"b3ed718d-5c96-403d-85c2-6e1cbca41b58\") " pod="openshift-network-operator/iptables-alerter-xpklc"
Sep 08 14:34:11.664726 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664537    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hppsk\" (UniqueName: \"kubernetes.io/projected/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-kube-api-access-hppsk\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.664726 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664580    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-var-lib-openvswitch\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.664726 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664610    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-run-ovn\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.664726 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664631    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-sysconfig\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.664726 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664677    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 14:34:11.664726 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664672    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-sysctl-conf\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664730    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-lib-modules\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664753    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6-agent-certs\") pod \"konnectivity-agent-fjchd\" (UID: \"b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6\") " pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664775    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-sysctl-d\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664797    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-registration-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664850    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22348c24-e76f-4cd1-8646-7f3338b45ec7-dbus\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664907    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-run-systemd\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664929    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55f744b4-526c-4826-8bc5-46ace246b9d7-ovnkube-config\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664959    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55f744b4-526c-4826-8bc5-46ace246b9d7-ovn-node-metrics-cert\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664971    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78adf8d2-113a-4251-a33f-66236876be03-system-cni-dir\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.664983    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/78adf8d2-113a-4251-a33f-66236876be03-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665002    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/78adf8d2-113a-4251-a33f-66236876be03-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.665022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665016    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55f744b4-526c-4826-8bc5-46ace246b9d7-ovnkube-script-lib\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665049    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6-konnectivity-ca\") pod \"konnectivity-agent-fjchd\" (UID: \"b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6\") " pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665077    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-etc-openvswitch\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665099    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-log-socket\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665129    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665154    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-modprobe-d\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665208    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zp9h\" (UniqueName: \"kubernetes.io/projected/b3ed718d-5c96-403d-85c2-6e1cbca41b58-kube-api-access-5zp9h\") pod \"iptables-alerter-xpklc\" (UID: \"b3ed718d-5c96-403d-85c2-6e1cbca41b58\") " pod="openshift-network-operator/iptables-alerter-xpklc"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665227    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-kubelet\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665296    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-tuned\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665321    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78adf8d2-113a-4251-a33f-66236876be03-cni-binary-copy\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665339    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-socket-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665355    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-slash\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.665421 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665396    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d88ea008-9ce7-4a83-abd8-bbde485345a3-hosts-file\") pod \"node-resolver-vsxtr\" (UID: \"d88ea008-9ce7-4a83-abd8-bbde485345a3\") " pod="openshift-dns/node-resolver-vsxtr"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665429    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2mf6\" (UniqueName: \"kubernetes.io/projected/78adf8d2-113a-4251-a33f-66236876be03-kube-api-access-p2mf6\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665453    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665476    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-sys\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665497    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjrgw\" (UniqueName: \"kubernetes.io/projected/2a15e5b5-3ba1-4587-943a-a4989e67b57b-kube-api-access-hjrgw\") pod \"node-ca-hvzkl\" (UID: \"2a15e5b5-3ba1-4587-943a-a4989e67b57b\") " pod="openshift-image-registry/node-ca-hvzkl"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665535    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j75gz\" (UniqueName: \"kubernetes.io/projected/55f744b4-526c-4826-8bc5-46ace246b9d7-kube-api-access-j75gz\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665572    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-systemd\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665604    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b3190eb-4813-4311-bbcc-a2abc50417bf-tmp\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665632    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22348c24-e76f-4cd1-8646-7f3338b45ec7-kubelet-config\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665652    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f2md\" (UniqueName: \"kubernetes.io/projected/4b3190eb-4813-4311-bbcc-a2abc50417bf-kube-api-access-7f2md\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665672    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a15e5b5-3ba1-4587-943a-a4989e67b57b-serviceca\") pod \"node-ca-hvzkl\" (UID: \"2a15e5b5-3ba1-4587-943a-a4989e67b57b\") " pod="openshift-image-registry/node-ca-hvzkl"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665700    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.665782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.665718    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-sys-fs\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.673025 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:11.673006    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358c2d635fdf18050ee800796677d30a.slice/crio-0acf49266f421b96e4c8796d6fc7c1ba2718bd62e8e8c4f6ff90bc401de8a4bb WatchSource:0}: Error finding container 0acf49266f421b96e4c8796d6fc7c1ba2718bd62e8e8c4f6ff90bc401de8a4bb: Status 404 returned error can't find the container with id 0acf49266f421b96e4c8796d6fc7c1ba2718bd62e8e8c4f6ff90bc401de8a4bb
Sep 08 14:34:11.686728 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.686713    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dwj4x"
Sep 08 14:34:11.695047 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.695030    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dwj4x"
Sep 08 14:34:11.751880 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.751862    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 14:34:11.765955 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.765933    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-tuned\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.766043 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.765956    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78adf8d2-113a-4251-a33f-66236876be03-cni-binary-copy\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.766043 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.765971    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-socket-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.766043 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.765983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-slash\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.766043 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766020    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-var-lib-kubelet\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.766043 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766035    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d88ea008-9ce7-4a83-abd8-bbde485345a3-hosts-file\") pod \"node-resolver-vsxtr\" (UID: \"d88ea008-9ce7-4a83-abd8-bbde485345a3\") " pod="openshift-dns/node-resolver-vsxtr"
Sep 08 14:34:11.766268 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766047    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2mf6\" (UniqueName: \"kubernetes.io/projected/78adf8d2-113a-4251-a33f-66236876be03-kube-api-access-p2mf6\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.766268 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766084    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.766268 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766108    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-run-netns\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.766268 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766137    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.766268 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766135    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/328b6c1c-4a50-456f-9da9-795eba13313f-multus-daemon-config\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.766268 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766178    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d88ea008-9ce7-4a83-abd8-bbde485345a3-hosts-file\") pod \"node-resolver-vsxtr\" (UID: \"d88ea008-9ce7-4a83-abd8-bbde485345a3\") " pod="openshift-dns/node-resolver-vsxtr"
Sep 08 14:34:11.766268 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-sys\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.766268 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766086    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-slash\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.766268 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766252    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-sys\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.766268 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766260    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjrgw\" (UniqueName: \"kubernetes.io/projected/2a15e5b5-3ba1-4587-943a-a4989e67b57b-kube-api-access-hjrgw\") pod \"node-ca-hvzkl\" (UID: \"2a15e5b5-3ba1-4587-943a-a4989e67b57b\") " pod="openshift-image-registry/node-ca-hvzkl"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766282    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j75gz\" (UniqueName: \"kubernetes.io/projected/55f744b4-526c-4826-8bc5-46ace246b9d7-kube-api-access-j75gz\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766303    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-etc-kubernetes\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766347    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-systemd\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766363    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b3190eb-4813-4311-bbcc-a2abc50417bf-tmp\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766395    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22348c24-e76f-4cd1-8646-7f3338b45ec7-kubelet-config\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766421    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-cnibin\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766446    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-multus-conf-dir\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766455    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-systemd\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766472    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f2md\" (UniqueName: \"kubernetes.io/projected/4b3190eb-4813-4311-bbcc-a2abc50417bf-kube-api-access-7f2md\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a15e5b5-3ba1-4587-943a-a4989e67b57b-serviceca\") pod \"node-ca-hvzkl\" (UID: \"2a15e5b5-3ba1-4587-943a-a4989e67b57b\") " pod="openshift-image-registry/node-ca-hvzkl"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766579    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766603    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-socket-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766617    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-sys-fs\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766642    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/328b6c1c-4a50-456f-9da9-795eba13313f-cni-binary-copy\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766664    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766670    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grbd\" (UniqueName: \"kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd\") pod \"network-check-target-php6q\" (UID: \"ac6d150a-3459-4e57-93c5-2c52ec02c6a3\") " pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:11.766690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766620    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22348c24-e76f-4cd1-8646-7f3338b45ec7-kubelet-config\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766703    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-var-lib-kubelet\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766727    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-device-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766749    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-var-lib-kubelet\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766687    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-sys-fs\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766754    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766807    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-device-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766860    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-systemd-units\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-run-openvswitch\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:11.766904    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55f744b4-526c-4826-8bc5-46ace246b9d7-env-overrides\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766945    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-run-openvswitch\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766949    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a15e5b5-3ba1-4587-943a-a4989e67b57b-host\") pod \"node-ca-hvzkl\" (UID: \"2a15e5b5-3ba1-4587-943a-a4989e67b57b\") " pod="openshift-image-registry/node-ca-hvzkl"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766959    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-systemd-units\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766960    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78adf8d2-113a-4251-a33f-66236876be03-cni-binary-copy\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766992    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a15e5b5-3ba1-4587-943a-a4989e67b57b-host\") pod \"node-ca-hvzkl\" (UID: \"2a15e5b5-3ba1-4587-943a-a4989e67b57b\") " pod="openshift-image-registry/node-ca-hvzkl"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.766860    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 14:34:11.767489 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:11.767075    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret podName:22348c24-e76f-4cd1-8646-7f3338b45ec7 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:12.267054215 +0000 UTC m=+2.994871219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret") pod "global-pull-secret-syncer-ntzrt" (UID: "22348c24-e76f-4cd1-8646-7f3338b45ec7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767067    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a15e5b5-3ba1-4587-943a-a4989e67b57b-serviceca\") pod \"node-ca-hvzkl\" (UID: \"2a15e5b5-3ba1-4587-943a-a4989e67b57b\") " pod="openshift-image-registry/node-ca-hvzkl"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767066    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-run\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767108    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d88ea008-9ce7-4a83-abd8-bbde485345a3-tmp-dir\") pod \"node-resolver-vsxtr\" (UID: \"d88ea008-9ce7-4a83-abd8-bbde485345a3\") " pod="openshift-dns/node-resolver-vsxtr"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767130    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kpsj\" (UniqueName: \"kubernetes.io/projected/d88ea008-9ce7-4a83-abd8-bbde485345a3-kube-api-access-9kpsj\") pod \"node-resolver-vsxtr\" (UID: \"d88ea008-9ce7-4a83-abd8-bbde485345a3\") " pod="openshift-dns/node-resolver-vsxtr"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767189    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78adf8d2-113a-4251-a33f-66236876be03-cnibin\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767204    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-run\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767210    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-run-netns\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767236    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-run-k8s-cni-cncf-io\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767259    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767261    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78adf8d2-113a-4251-a33f-66236876be03-cnibin\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767295    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-run-netns\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767327    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-kubernetes\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3ed718d-5c96-403d-85c2-6e1cbca41b58-iptables-alerter-script\") pod \"iptables-alerter-xpklc\" (UID: \"b3ed718d-5c96-403d-85c2-6e1cbca41b58\") " pod="openshift-network-operator/iptables-alerter-xpklc"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767371    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55f744b4-526c-4826-8bc5-46ace246b9d7-env-overrides\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767371    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d88ea008-9ce7-4a83-abd8-bbde485345a3-tmp-dir\") pod \"node-resolver-vsxtr\" (UID: \"d88ea008-9ce7-4a83-abd8-bbde485345a3\") " pod="openshift-dns/node-resolver-vsxtr"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767381    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78adf8d2-113a-4251-a33f-66236876be03-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.768321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767438    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-node-log\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767451    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-kubernetes\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767461    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-cni-bin\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767489    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-cni-netd\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767513    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-node-log\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767502    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-cni-bin\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767529    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-multus-cni-dir\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767549    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-multus-socket-dir-parent\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767556    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-cni-netd\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767573    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-host\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767607    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-host\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-etc-selinux\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-hostroot\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767666    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kplv\" (UniqueName: \"kubernetes.io/projected/328b6c1c-4a50-456f-9da9-795eba13313f-kube-api-access-8kplv\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767667    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78adf8d2-113a-4251-a33f-66236876be03-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767690    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3ed718d-5c96-403d-85c2-6e1cbca41b58-host-slash\") pod \"iptables-alerter-xpklc\" (UID: \"b3ed718d-5c96-403d-85c2-6e1cbca41b58\") " pod="openshift-network-operator/iptables-alerter-xpklc"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767695    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-etc-selinux\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.769894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hppsk\" (UniqueName: \"kubernetes.io/projected/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-kube-api-access-hppsk\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767721    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3ed718d-5c96-403d-85c2-6e1cbca41b58-host-slash\") pod \"iptables-alerter-xpklc\" (UID: \"b3ed718d-5c96-403d-85c2-6e1cbca41b58\") " pod="openshift-network-operator/iptables-alerter-xpklc"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767740    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-var-lib-openvswitch\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767772    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-run-ovn\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767806    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-var-lib-openvswitch\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767844    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-run-ovn\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767864    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-sysconfig\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767879    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-sysctl-conf\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767893    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-lib-modules\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767950    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3ed718d-5c96-403d-85c2-6e1cbca41b58-iptables-alerter-script\") pod \"iptables-alerter-xpklc\" (UID: \"b3ed718d-5c96-403d-85c2-6e1cbca41b58\") " pod="openshift-network-operator/iptables-alerter-xpklc"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767956    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-sysconfig\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767983    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-lib-modules\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.767997    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6-agent-certs\") pod \"konnectivity-agent-fjchd\" (UID: \"b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6\") " pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768019    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-sysctl-conf\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768021    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-system-cni-dir\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768088    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-sysctl-d\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768144    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-registration-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.770605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768159    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-sysctl-d\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768167    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22348c24-e76f-4cd1-8646-7f3338b45ec7-dbus\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768198    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-run-systemd\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768202    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-registration-dir\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768228    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55f744b4-526c-4826-8bc5-46ace246b9d7-ovnkube-config\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768252    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-run-systemd\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768265    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55f744b4-526c-4826-8bc5-46ace246b9d7-ovn-node-metrics-cert\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768312    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22348c24-e76f-4cd1-8646-7f3338b45ec7-dbus\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768294    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78adf8d2-113a-4251-a33f-66236876be03-system-cni-dir\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768355    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78adf8d2-113a-4251-a33f-66236876be03-system-cni-dir\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768351    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/78adf8d2-113a-4251-a33f-66236876be03-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768384    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/78adf8d2-113a-4251-a33f-66236876be03-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768398    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55f744b4-526c-4826-8bc5-46ace246b9d7-ovnkube-script-lib\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768418    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-var-lib-cni-multus\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768623    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6-konnectivity-ca\") pod \"konnectivity-agent-fjchd\" (UID: \"b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6\") " pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768680    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-etc-openvswitch\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-log-socket\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768770    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-modprobe-d\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768953    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55f744b4-526c-4826-8bc5-46ace246b9d7-ovnkube-config\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.768999    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zp9h\" (UniqueName: \"kubernetes.io/projected/b3ed718d-5c96-403d-85c2-6e1cbca41b58-kube-api-access-5zp9h\") pod \"iptables-alerter-xpklc\" (UID: \"b3ed718d-5c96-403d-85c2-6e1cbca41b58\") " pod="openshift-network-operator/iptables-alerter-xpklc"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769020    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-kubelet\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769035    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-log-socket\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769053    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-var-lib-cni-bin\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769073    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-run-multus-certs\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769081    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-etc-openvswitch\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769127    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzc5b\" (UniqueName: \"kubernetes.io/projected/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-kube-api-access-wzc5b\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769246    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-kubelet\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769333    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55f744b4-526c-4826-8bc5-46ace246b9d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769345    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/78adf8d2-113a-4251-a33f-66236876be03-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769376    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-modprobe-d\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769547    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/78adf8d2-113a-4251-a33f-66236876be03-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769661    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6-konnectivity-ca\") pod \"konnectivity-agent-fjchd\" (UID: \"b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6\") " pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.769729    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55f744b4-526c-4826-8bc5-46ace246b9d7-ovnkube-script-lib\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.771712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.770959    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b3190eb-4813-4311-bbcc-a2abc50417bf-etc-tuned\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.772203 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.771022    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b3190eb-4813-4311-bbcc-a2abc50417bf-tmp\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.772203 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.771073    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55f744b4-526c-4826-8bc5-46ace246b9d7-ovn-node-metrics-cert\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.772203 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.771090    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6-agent-certs\") pod \"konnectivity-agent-fjchd\" (UID: \"b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6\") " pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:11.778728 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.778711    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kpsj\" (UniqueName: \"kubernetes.io/projected/d88ea008-9ce7-4a83-abd8-bbde485345a3-kube-api-access-9kpsj\") pod \"node-resolver-vsxtr\" (UID: \"d88ea008-9ce7-4a83-abd8-bbde485345a3\") " pod="openshift-dns/node-resolver-vsxtr"
Sep 08 14:34:11.780669 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.780649    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j75gz\" (UniqueName: \"kubernetes.io/projected/55f744b4-526c-4826-8bc5-46ace246b9d7-kube-api-access-j75gz\") pod \"ovnkube-node-9lt7h\" (UID: \"55f744b4-526c-4826-8bc5-46ace246b9d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.782073 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.781939    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zp9h\" (UniqueName: \"kubernetes.io/projected/b3ed718d-5c96-403d-85c2-6e1cbca41b58-kube-api-access-5zp9h\") pod \"iptables-alerter-xpklc\" (UID: \"b3ed718d-5c96-403d-85c2-6e1cbca41b58\") " pod="openshift-network-operator/iptables-alerter-xpklc"
Sep 08 14:34:11.782073 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:11.781997    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:34:11.782073 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:11.782024    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:34:11.782073 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:11.782068    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8grbd for pod openshift-network-diagnostics/network-check-target-php6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:11.784874 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:11.782466    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd podName:ac6d150a-3459-4e57-93c5-2c52ec02c6a3 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:12.282444053 +0000 UTC m=+3.010261062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8grbd" (UniqueName: "kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd") pod "network-check-target-php6q" (UID: "ac6d150a-3459-4e57-93c5-2c52ec02c6a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:11.784874 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.783904    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f2md\" (UniqueName: \"kubernetes.io/projected/4b3190eb-4813-4311-bbcc-a2abc50417bf-kube-api-access-7f2md\") pod \"tuned-qljrl\" (UID: \"4b3190eb-4813-4311-bbcc-a2abc50417bf\") " pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.784874 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.784091    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2mf6\" (UniqueName: \"kubernetes.io/projected/78adf8d2-113a-4251-a33f-66236876be03-kube-api-access-p2mf6\") pod \"multus-additional-cni-plugins-5wbz8\" (UID: \"78adf8d2-113a-4251-a33f-66236876be03\") " pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.785443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.785423    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjrgw\" (UniqueName: \"kubernetes.io/projected/2a15e5b5-3ba1-4587-943a-a4989e67b57b-kube-api-access-hjrgw\") pod \"node-ca-hvzkl\" (UID: \"2a15e5b5-3ba1-4587-943a-a4989e67b57b\") " pod="openshift-image-registry/node-ca-hvzkl"
Sep 08 14:34:11.785510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.785479    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hppsk\" (UniqueName: \"kubernetes.io/projected/c874f8e7-a9b5-44dc-bdde-4b21dff86a47-kube-api-access-hppsk\") pod \"aws-ebs-csi-driver-node-b4wt2\" (UID: \"c874f8e7-a9b5-44dc-bdde-4b21dff86a47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.790709 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.790672    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-6.ec2.internal" event={"ID":"358c2d635fdf18050ee800796677d30a","Type":"ContainerStarted","Data":"0acf49266f421b96e4c8796d6fc7c1ba2718bd62e8e8c4f6ff90bc401de8a4bb"}
Sep 08 14:34:11.791616 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.791579    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal" event={"ID":"15f573fb3bfde2f504806a4ebee87ad0","Type":"ContainerStarted","Data":"c826b1d33830a404a6b2ad1ae992b86a2b85cc07b6f0100407f751a8ca025e51"}
Sep 08 14:34:11.870265 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870232    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-var-lib-cni-multus\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870274    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-var-lib-cni-bin\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870297    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-run-multus-certs\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870318    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzc5b\" (UniqueName: \"kubernetes.io/projected/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-kube-api-access-wzc5b\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:11.870394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870324    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-var-lib-cni-multus\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870348    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-var-lib-kubelet\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870375    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-run-netns\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870379    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-run-multus-certs\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870397    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/328b6c1c-4a50-456f-9da9-795eba13313f-multus-daemon-config\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870406    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-var-lib-cni-bin\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870430    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-etc-kubernetes\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870452    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-cnibin\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-multus-conf-dir\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870482    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/328b6c1c-4a50-456f-9da9-795eba13313f-cni-binary-copy\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870528    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-run-k8s-cni-cncf-io\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870542    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870558    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-multus-cni-dir\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870571    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-multus-socket-dir-parent\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870608    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-hostroot\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870607    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-var-lib-kubelet\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870623    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kplv\" (UniqueName: \"kubernetes.io/projected/328b6c1c-4a50-456f-9da9-795eba13313f-kube-api-access-8kplv\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870643    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-system-cni-dir\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870650    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-run-netns\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.870715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870695    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-multus-conf-dir\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.871443 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:11.870753    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:11.871443 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:11.870797    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs podName:db5c60b0-c15e-496d-b2a0-8c76704ee8ae nodeName:}" failed. No retries permitted until 2026-09-08 14:34:12.370785319 +0000 UTC m=+3.098602310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs") pod "network-metrics-daemon-tmqvh" (UID: "db5c60b0-c15e-496d-b2a0-8c76704ee8ae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:11.871443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870916    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-etc-kubernetes\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.871443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870928    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-system-cni-dir\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.871443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870986    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-cnibin\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.871443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870983    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-hostroot\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.871443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.870999    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-multus-cni-dir\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.871443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.871048    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-host-run-k8s-cni-cncf-io\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.871443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.871118    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/328b6c1c-4a50-456f-9da9-795eba13313f-multus-socket-dir-parent\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.871443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.871131    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/328b6c1c-4a50-456f-9da9-795eba13313f-cni-binary-copy\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.871443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.871353    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/328b6c1c-4a50-456f-9da9-795eba13313f-multus-daemon-config\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.886568 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.886543    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kplv\" (UniqueName: \"kubernetes.io/projected/328b6c1c-4a50-456f-9da9-795eba13313f-kube-api-access-8kplv\") pod \"multus-dn9wg\" (UID: \"328b6c1c-4a50-456f-9da9-795eba13313f\") " pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:11.886688 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.886570    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzc5b\" (UniqueName: \"kubernetes.io/projected/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-kube-api-access-wzc5b\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:11.952227 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.952203    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5wbz8"
Sep 08 14:34:11.960280 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.960252    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2"
Sep 08 14:34:11.961422 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:11.961402    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78adf8d2_113a_4251_a33f_66236876be03.slice/crio-a88878ea5e519c7e58cb9ea6a4fc06417e6bd42b2ef539fcfd3d0c21ca39d42d WatchSource:0}: Error finding container a88878ea5e519c7e58cb9ea6a4fc06417e6bd42b2ef539fcfd3d0c21ca39d42d: Status 404 returned error can't find the container with id a88878ea5e519c7e58cb9ea6a4fc06417e6bd42b2ef539fcfd3d0c21ca39d42d
Sep 08 14:34:11.965888 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.965866    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qljrl"
Sep 08 14:34:11.971040 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.971021    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vsxtr"
Sep 08 14:34:11.974319 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:11.974300    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc874f8e7_a9b5_44dc_bdde_4b21dff86a47.slice/crio-72deccac765329ca518554a149893c2e097c4707f655f099a50aee5b1ffb2df2 WatchSource:0}: Error finding container 72deccac765329ca518554a149893c2e097c4707f655f099a50aee5b1ffb2df2: Status 404 returned error can't find the container with id 72deccac765329ca518554a149893c2e097c4707f655f099a50aee5b1ffb2df2
Sep 08 14:34:11.974877 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:11.974858    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b3190eb_4813_4311_bbcc_a2abc50417bf.slice/crio-ab3826b08e13e2367cb126958a8e8cec7553956e83acb81e46144c22d6e58237 WatchSource:0}: Error finding container ab3826b08e13e2367cb126958a8e8cec7553956e83acb81e46144c22d6e58237: Status 404 returned error can't find the container with id ab3826b08e13e2367cb126958a8e8cec7553956e83acb81e46144c22d6e58237
Sep 08 14:34:11.975522 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.975502    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xpklc"
Sep 08 14:34:11.980919 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.980903    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:11.982471 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:11.982450    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd88ea008_9ce7_4a83_abd8_bbde485345a3.slice/crio-2963b6f30b08b03a4f7dc943564afe4fb39d26916d148b3cdadc66074e6c038f WatchSource:0}: Error finding container 2963b6f30b08b03a4f7dc943564afe4fb39d26916d148b3cdadc66074e6c038f: Status 404 returned error can't find the container with id 2963b6f30b08b03a4f7dc943564afe4fb39d26916d148b3cdadc66074e6c038f
Sep 08 14:34:11.985761 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:11.985507    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ed718d_5c96_403d_85c2_6e1cbca41b58.slice/crio-c447d70bdd341f97ea9f15f579880422996d29de0565e05b05f8da5b4bb327cd WatchSource:0}: Error finding container c447d70bdd341f97ea9f15f579880422996d29de0565e05b05f8da5b4bb327cd: Status 404 returned error can't find the container with id c447d70bdd341f97ea9f15f579880422996d29de0565e05b05f8da5b4bb327cd
Sep 08 14:34:11.986362 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.986216    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:11.989880 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:11.989860    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d17862_2d5a_492b_a3e8_eb2b8c5eb2c6.slice/crio-4cc9970cb2c345c2bb22a3118721848be3e958dcd6e3e0870f49ad55da98872e WatchSource:0}: Error finding container 4cc9970cb2c345c2bb22a3118721848be3e958dcd6e3e0870f49ad55da98872e: Status 404 returned error can't find the container with id 4cc9970cb2c345c2bb22a3118721848be3e958dcd6e3e0870f49ad55da98872e
Sep 08 14:34:11.991036 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.991020    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hvzkl"
Sep 08 14:34:11.994733 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:11.994716    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55f744b4_526c_4826_8bc5_46ace246b9d7.slice/crio-239fd13c34b74497a7d5e49627b758fa8965419be6a443950e9f209d50e4afe5 WatchSource:0}: Error finding container 239fd13c34b74497a7d5e49627b758fa8965419be6a443950e9f209d50e4afe5: Status 404 returned error can't find the container with id 239fd13c34b74497a7d5e49627b758fa8965419be6a443950e9f209d50e4afe5
Sep 08 14:34:11.995401 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:11.995384    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dn9wg"
Sep 08 14:34:12.003736 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:12.003717    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a15e5b5_3ba1_4587_943a_a4989e67b57b.slice/crio-7785cbd9bf22127bef3e10bd65db75f3d4625fe68b044a7aa886b74845add4cf WatchSource:0}: Error finding container 7785cbd9bf22127bef3e10bd65db75f3d4625fe68b044a7aa886b74845add4cf: Status 404 returned error can't find the container with id 7785cbd9bf22127bef3e10bd65db75f3d4625fe68b044a7aa886b74845add4cf
Sep 08 14:34:12.006528 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:12.006509    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328b6c1c_4a50_456f_9da9_795eba13313f.slice/crio-9143f5f7ddadd130371f63ea78c1f928b016e3bcab16e9e013ec3f2f471b8efd WatchSource:0}: Error finding container 9143f5f7ddadd130371f63ea78c1f928b016e3bcab16e9e013ec3f2f471b8efd: Status 404 returned error can't find the container with id 9143f5f7ddadd130371f63ea78c1f928b016e3bcab16e9e013ec3f2f471b8efd
Sep 08 14:34:12.272966 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.272930    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:12.273110 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:12.273094    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:12.273177 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:12.273144    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret podName:22348c24-e76f-4cd1-8646-7f3338b45ec7 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:13.273131605 +0000 UTC m=+4.000948599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret") pod "global-pull-secret-syncer-ntzrt" (UID: "22348c24-e76f-4cd1-8646-7f3338b45ec7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:12.373594 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.373564    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grbd\" (UniqueName: \"kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd\") pod \"network-check-target-php6q\" (UID: \"ac6d150a-3459-4e57-93c5-2c52ec02c6a3\") " pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:12.373775 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.373629    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:12.373775 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:12.373753    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:12.373920 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:12.373805    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:34:12.373920 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:12.373843    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs podName:db5c60b0-c15e-496d-b2a0-8c76704ee8ae nodeName:}" failed. No retries permitted until 2026-09-08 14:34:13.373808231 +0000 UTC m=+4.101625239 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs") pod "network-metrics-daemon-tmqvh" (UID: "db5c60b0-c15e-496d-b2a0-8c76704ee8ae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:12.373920 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:12.373845    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:34:12.373920 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:12.373860    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8grbd for pod openshift-network-diagnostics/network-check-target-php6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:12.373920 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:12.373889    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd podName:ac6d150a-3459-4e57-93c5-2c52ec02c6a3 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:13.373882586 +0000 UTC m=+4.101699576 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8grbd" (UniqueName: "kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd") pod "network-check-target-php6q" (UID: "ac6d150a-3459-4e57-93c5-2c52ec02c6a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:12.478786 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.478748    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 14:34:12.697298 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.697180    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 14:29:11 +0000 UTC" deadline="2028-05-06 17:53:52.592838921 +0000 UTC"
Sep 08 14:34:12.697298 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.697246    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14547h19m39.895597387s"
Sep 08 14:34:12.794959 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.794919    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hvzkl" event={"ID":"2a15e5b5-3ba1-4587-943a-a4989e67b57b","Type":"ContainerStarted","Data":"7785cbd9bf22127bef3e10bd65db75f3d4625fe68b044a7aa886b74845add4cf"}
Sep 08 14:34:12.796611 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.796579    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h" event={"ID":"55f744b4-526c-4826-8bc5-46ace246b9d7","Type":"ContainerStarted","Data":"239fd13c34b74497a7d5e49627b758fa8965419be6a443950e9f209d50e4afe5"}
Sep 08 14:34:12.798470 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.798444    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fjchd" event={"ID":"b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6","Type":"ContainerStarted","Data":"4cc9970cb2c345c2bb22a3118721848be3e958dcd6e3e0870f49ad55da98872e"}
Sep 08 14:34:12.800086 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.800062    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xpklc" event={"ID":"b3ed718d-5c96-403d-85c2-6e1cbca41b58","Type":"ContainerStarted","Data":"c447d70bdd341f97ea9f15f579880422996d29de0565e05b05f8da5b4bb327cd"}
Sep 08 14:34:12.802450 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.802427    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2" event={"ID":"c874f8e7-a9b5-44dc-bdde-4b21dff86a47","Type":"ContainerStarted","Data":"72deccac765329ca518554a149893c2e097c4707f655f099a50aee5b1ffb2df2"}
Sep 08 14:34:12.804766 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.804741    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dn9wg" event={"ID":"328b6c1c-4a50-456f-9da9-795eba13313f","Type":"ContainerStarted","Data":"9143f5f7ddadd130371f63ea78c1f928b016e3bcab16e9e013ec3f2f471b8efd"}
Sep 08 14:34:12.809059 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.809004    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vsxtr" event={"ID":"d88ea008-9ce7-4a83-abd8-bbde485345a3","Type":"ContainerStarted","Data":"2963b6f30b08b03a4f7dc943564afe4fb39d26916d148b3cdadc66074e6c038f"}
Sep 08 14:34:12.811024 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.810867    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qljrl" event={"ID":"4b3190eb-4813-4311-bbcc-a2abc50417bf","Type":"ContainerStarted","Data":"ab3826b08e13e2367cb126958a8e8cec7553956e83acb81e46144c22d6e58237"}
Sep 08 14:34:12.813629 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:12.813575    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wbz8" event={"ID":"78adf8d2-113a-4251-a33f-66236876be03","Type":"ContainerStarted","Data":"a88878ea5e519c7e58cb9ea6a4fc06417e6bd42b2ef539fcfd3d0c21ca39d42d"}
Sep 08 14:34:13.283326 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:13.283289    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:13.283578 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:13.283457    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:13.283578 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:13.283521    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret podName:22348c24-e76f-4cd1-8646-7f3338b45ec7 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:15.283502974 +0000 UTC m=+6.011319981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret") pod "global-pull-secret-syncer-ntzrt" (UID: "22348c24-e76f-4cd1-8646-7f3338b45ec7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:13.383847 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:13.383763    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grbd\" (UniqueName: \"kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd\") pod \"network-check-target-php6q\" (UID: \"ac6d150a-3459-4e57-93c5-2c52ec02c6a3\") " pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:13.383847 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:13.383816    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:13.384098 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:13.383963    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:13.384098 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:13.383994    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:34:13.384098 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:13.384018    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:34:13.384098 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:13.384023    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs podName:db5c60b0-c15e-496d-b2a0-8c76704ee8ae nodeName:}" failed. No retries permitted until 2026-09-08 14:34:15.384004996 +0000 UTC m=+6.111821988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs") pod "network-metrics-daemon-tmqvh" (UID: "db5c60b0-c15e-496d-b2a0-8c76704ee8ae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:13.384098 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:13.384031    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8grbd for pod openshift-network-diagnostics/network-check-target-php6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:13.384098 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:13.384085    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd podName:ac6d150a-3459-4e57-93c5-2c52ec02c6a3 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:15.384069924 +0000 UTC m=+6.111886917 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8grbd" (UniqueName: "kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd") pod "network-check-target-php6q" (UID: "ac6d150a-3459-4e57-93c5-2c52ec02c6a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:13.697769 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:13.697684    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 14:29:11 +0000 UTC" deadline="2028-02-19 16:18:54.259287097 +0000 UTC"
Sep 08 14:34:13.697769 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:13.697723    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12697h44m40.561569169s"
Sep 08 14:34:13.788735 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:13.788707    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:13.788735 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:13.788722    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:13.788983 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:13.788818    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:13.788983 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:13.788890    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:13.788983 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:13.788945    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:13.789128 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:13.789014    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:15.299101 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:15.298519    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:15.299101 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:15.298683    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:15.299101 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:15.298744    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret podName:22348c24-e76f-4cd1-8646-7f3338b45ec7 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:19.298726616 +0000 UTC m=+10.026543623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret") pod "global-pull-secret-syncer-ntzrt" (UID: "22348c24-e76f-4cd1-8646-7f3338b45ec7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:15.399401 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:15.399363    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grbd\" (UniqueName: \"kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd\") pod \"network-check-target-php6q\" (UID: \"ac6d150a-3459-4e57-93c5-2c52ec02c6a3\") " pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:15.399578 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:15.399437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:15.399578 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:15.399570    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:15.399657 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:15.399629    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs podName:db5c60b0-c15e-496d-b2a0-8c76704ee8ae nodeName:}" failed. No retries permitted until 2026-09-08 14:34:19.399615274 +0000 UTC m=+10.127432279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs") pod "network-metrics-daemon-tmqvh" (UID: "db5c60b0-c15e-496d-b2a0-8c76704ee8ae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:15.400021 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:15.400003    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:34:15.400095 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:15.400028    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:34:15.400095 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:15.400042    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8grbd for pod openshift-network-diagnostics/network-check-target-php6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:15.400095 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:15.400091    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd podName:ac6d150a-3459-4e57-93c5-2c52ec02c6a3 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:19.40007581 +0000 UTC m=+10.127892818 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8grbd" (UniqueName: "kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd") pod "network-check-target-php6q" (UID: "ac6d150a-3459-4e57-93c5-2c52ec02c6a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:15.789065 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:15.788524    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:15.789065 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:15.788652    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:15.789065 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:15.788921    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:15.789065 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:15.789025    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:15.789401 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:15.789116    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:15.789401 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:15.789229    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:17.787418 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:17.787333    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:17.787866 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:17.787333    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:17.787866 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:17.787464    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:17.787866 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:17.787544    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:17.787866 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:17.787333    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:17.787866 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:17.787639    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:19.335384 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:19.335299    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:19.335777 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:19.335416    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:19.335777 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:19.335528    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret podName:22348c24-e76f-4cd1-8646-7f3338b45ec7 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:27.335505545 +0000 UTC m=+18.063322540 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret") pod "global-pull-secret-syncer-ntzrt" (UID: "22348c24-e76f-4cd1-8646-7f3338b45ec7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:19.436350 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:19.436313    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grbd\" (UniqueName: \"kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd\") pod \"network-check-target-php6q\" (UID: \"ac6d150a-3459-4e57-93c5-2c52ec02c6a3\") " pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:19.436525 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:19.436382    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:19.436525 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:19.436516    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:19.436648 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:19.436572    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs podName:db5c60b0-c15e-496d-b2a0-8c76704ee8ae nodeName:}" failed. No retries permitted until 2026-09-08 14:34:27.436557766 +0000 UTC m=+18.164374761 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs") pod "network-metrics-daemon-tmqvh" (UID: "db5c60b0-c15e-496d-b2a0-8c76704ee8ae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:19.436986 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:19.436965    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:34:19.436986 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:19.436989    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:34:19.437138 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:19.437002    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8grbd for pod openshift-network-diagnostics/network-check-target-php6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:19.437138 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:19.437047    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd podName:ac6d150a-3459-4e57-93c5-2c52ec02c6a3 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:27.437033396 +0000 UTC m=+18.164850405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8grbd" (UniqueName: "kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd") pod "network-check-target-php6q" (UID: "ac6d150a-3459-4e57-93c5-2c52ec02c6a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:19.788509 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:19.788474    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:19.788659 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:19.788558    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:19.788659 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:19.788638    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:19.788786 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:19.788765    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:19.788865 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:19.788816    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:19.788945 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:19.788927    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:21.787856 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:21.787808    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:21.788396 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:21.787951    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:21.788396 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:21.788355    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:21.788489 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:21.788460    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:21.788621 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:21.788606    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:21.788704 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:21.788688    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:23.787156 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:23.787121    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:23.787156 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:23.787157    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:23.787630 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:23.787127    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:23.787630 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:23.787246    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:23.787630 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:23.787347    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:23.787630 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:23.787438    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:25.787643 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:25.787611    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:25.788083 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:25.787652    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:25.788083 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:25.787622    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:25.788083 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:25.787739    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:25.788083 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:25.787807    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:25.788083 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:25.787982    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:27.394311 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:27.394280    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:27.394765 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:27.394453    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:27.394765 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:27.394522    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret podName:22348c24-e76f-4cd1-8646-7f3338b45ec7 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:43.394501682 +0000 UTC m=+34.122318674 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret") pod "global-pull-secret-syncer-ntzrt" (UID: "22348c24-e76f-4cd1-8646-7f3338b45ec7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:27.494788 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:27.494753    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grbd\" (UniqueName: \"kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd\") pod \"network-check-target-php6q\" (UID: \"ac6d150a-3459-4e57-93c5-2c52ec02c6a3\") " pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:27.494987 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:27.494811    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:27.494987 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:27.494945    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:34:27.494987 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:27.494959    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:27.494987 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:27.494972    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:34:27.494987 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:27.494988    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8grbd for pod openshift-network-diagnostics/network-check-target-php6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:27.495246 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:27.495026    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs podName:db5c60b0-c15e-496d-b2a0-8c76704ee8ae nodeName:}" failed. No retries permitted until 2026-09-08 14:34:43.495006743 +0000 UTC m=+34.222823747 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs") pod "network-metrics-daemon-tmqvh" (UID: "db5c60b0-c15e-496d-b2a0-8c76704ee8ae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:27.495246 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:27.495040    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd podName:ac6d150a-3459-4e57-93c5-2c52ec02c6a3 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:43.49503346 +0000 UTC m=+34.222850452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8grbd" (UniqueName: "kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd") pod "network-check-target-php6q" (UID: "ac6d150a-3459-4e57-93c5-2c52ec02c6a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:27.787439 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:27.787275    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:27.787439 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:27.787292    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:27.787439 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:27.787279    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:27.787439 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:27.787397    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:27.787786 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:27.787489    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:27.787786 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:27.787583    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:29.788651 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:29.788616    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:29.789129 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:29.788727    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:29.789209 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:29.789166    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:29.789288 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:29.789269    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:29.789365 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:29.789330    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:29.789421 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:29.789403    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:30.853376 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:30.853017    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h" event={"ID":"55f744b4-526c-4826-8bc5-46ace246b9d7","Type":"ContainerStarted","Data":"d34ee1282446c7e520958c74ef5bf6a816d5867dd433a71f38e2c326599b605f"}
Sep 08 14:34:30.853670 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:30.853468    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h" event={"ID":"55f744b4-526c-4826-8bc5-46ace246b9d7","Type":"ContainerStarted","Data":"f3c83420fe63145214f28178bc528acfafcd9de2dbaaf96693f12bd49e5022ac"}
Sep 08 14:34:30.854532 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:30.854506    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dn9wg" event={"ID":"328b6c1c-4a50-456f-9da9-795eba13313f","Type":"ContainerStarted","Data":"46ee03026618f0889a211b369e0733d01eb9754bab6b5b4b65ababfff65a501f"}
Sep 08 14:34:30.866747 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:30.866704    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qljrl" event={"ID":"4b3190eb-4813-4311-bbcc-a2abc50417bf","Type":"ContainerStarted","Data":"13dce9c8039a95ca7f2febc6ae3c300d0c4fa7d9f25805c1cff58e6910a10e71"}
Sep 08 14:34:30.874258 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:30.874234    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-6.ec2.internal" event={"ID":"358c2d635fdf18050ee800796677d30a","Type":"ContainerStarted","Data":"1096b5b38171e4866ca66d85989f168822b1421fc1961596dcd1d25262dfdaa1"}
Sep 08 14:34:30.875174 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:30.875114    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dn9wg" podStartSLOduration=3.51664158 podStartE2EDuration="21.875094729s" totalImagesPullingTime="18.358453149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:30.874652393 +0000 UTC m=+21.602469410" watchObservedRunningTime="2026-09-08 14:34:30.875094729 +0000 UTC m=+21.602911747"
Sep 08 14:34:30.909300 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:30.909257    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-6.ec2.internal" podStartSLOduration=20.909243233 podStartE2EDuration="20.909243233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:34:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:30.892867818 +0000 UTC m=+21.620684831" watchObservedRunningTime="2026-09-08 14:34:30.909243233 +0000 UTC m=+21.637060256"
Sep 08 14:34:30.909500 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:30.909469    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qljrl" podStartSLOduration=3.558581948 podStartE2EDuration="21.90945526s" totalImagesPullingTime="18.350873312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:30.90915903 +0000 UTC m=+21.636976129" watchObservedRunningTime="2026-09-08 14:34:30.90945526 +0000 UTC m=+21.637272273"
Sep 08 14:34:31.787121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.787087    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:31.787121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.787109    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:31.787395 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.787126    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:31.787395 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:31.787224    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:31.787395 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:31.787344    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:31.787503 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:31.787416    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:31.878752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.878725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fjchd" event={"ID":"b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6","Type":"ContainerStarted","Data":"252beaff825aa45d30a4b8083adc4e4d4e760a1be23ef5c5640d2a64bf793161"}
Sep 08 14:34:31.880286 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.880258    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xpklc" event={"ID":"b3ed718d-5c96-403d-85c2-6e1cbca41b58","Type":"ContainerStarted","Data":"3c3f10da638de2b9802f34704bf1dc6ea01fe2e7b28e19255fdf46b041c19845"}
Sep 08 14:34:31.881581 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.881562    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2" event={"ID":"c874f8e7-a9b5-44dc-bdde-4b21dff86a47","Type":"ContainerStarted","Data":"843fdc3b44dcc039ca470315d93e1cb2c64aa2ba3835320a0f898b433d1c4de6"}
Sep 08 14:34:31.883052 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.883030    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vsxtr" event={"ID":"d88ea008-9ce7-4a83-abd8-bbde485345a3","Type":"ContainerStarted","Data":"977cf10ee389edbff6fc6bb050164731de78b208760eb66c89bd87d0127162c4"}
Sep 08 14:34:31.884791 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.884770    2583 generic.go:356] "Generic (PLEG): container finished" podID="78adf8d2-113a-4251-a33f-66236876be03" containerID="70a3a3089418e440c08d47e7df2a8442fd229d8233f7540795f7feb958f7b3a2" exitCode=0
Sep 08 14:34:31.884920 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.884871    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wbz8" event={"ID":"78adf8d2-113a-4251-a33f-66236876be03","Type":"ContainerDied","Data":"70a3a3089418e440c08d47e7df2a8442fd229d8233f7540795f7feb958f7b3a2"}
Sep 08 14:34:31.886473 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.886450    2583 generic.go:356] "Generic (PLEG): container finished" podID="15f573fb3bfde2f504806a4ebee87ad0" containerID="a09dc872703f323c71da14464622a508fef7e807cd6c89828f0c0f62d677b4af" exitCode=0
Sep 08 14:34:31.886556 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.886544    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal" event={"ID":"15f573fb3bfde2f504806a4ebee87ad0","Type":"ContainerDied","Data":"a09dc872703f323c71da14464622a508fef7e807cd6c89828f0c0f62d677b4af"}
Sep 08 14:34:31.889017 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.888991    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hvzkl" event={"ID":"2a15e5b5-3ba1-4587-943a-a4989e67b57b","Type":"ContainerStarted","Data":"e8b55f304fc4ffed3f464e9a7a1951416beeef03b7f5aa6f4d8ed51d93aed203"}
Sep 08 14:34:31.894042 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.894024    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h" event={"ID":"55f744b4-526c-4826-8bc5-46ace246b9d7","Type":"ContainerStarted","Data":"d315e74dfbaee4fd4d797c76ff18d3f00f9891bbaa211c88da3bf76f8b5ae060"}
Sep 08 14:34:31.894145 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.894092    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h" event={"ID":"55f744b4-526c-4826-8bc5-46ace246b9d7","Type":"ContainerStarted","Data":"fe6d2dc01ccae0b88ff7ac643b939b2740947fe5cbf79090a4e762fa40b78a58"}
Sep 08 14:34:31.894207 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.894146    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h" event={"ID":"55f744b4-526c-4826-8bc5-46ace246b9d7","Type":"ContainerStarted","Data":"78fb12002a3557abece987b690115c48e4c2966f0f43bf958a143f0b2d102fb2"}
Sep 08 14:34:31.894207 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.894194    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h" event={"ID":"55f744b4-526c-4826-8bc5-46ace246b9d7","Type":"ContainerStarted","Data":"620758e970ae846aa3904142c8122938764339b85a9c1f9aeaec0a3ce336cd3e"}
Sep 08 14:34:31.919249 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.918660    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fjchd" podStartSLOduration=4.605032185 podStartE2EDuration="22.918642283s" totalImagesPullingTime="18.313610098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:31.898644953 +0000 UTC m=+22.626461969" watchObservedRunningTime="2026-09-08 14:34:31.918642283 +0000 UTC m=+22.646459298"
Sep 08 14:34:31.935655 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.935615    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vsxtr" podStartSLOduration=5.015897076 podStartE2EDuration="22.935599486s" totalImagesPullingTime="17.91970241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:31.919157076 +0000 UTC m=+22.646974086" watchObservedRunningTime="2026-09-08 14:34:31.935599486 +0000 UTC m=+22.663416496"
Sep 08 14:34:31.936083 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.936049    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hvzkl" podStartSLOduration=4.615031378 podStartE2EDuration="22.936040068s" totalImagesPullingTime="18.32100869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:31.93573937 +0000 UTC m=+22.663556383" watchObservedRunningTime="2026-09-08 14:34:31.936040068 +0000 UTC m=+22.663857083"
Sep 08 14:34:31.981284 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:31.981258    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 14:34:32.001083 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:32.001036    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xpklc" podStartSLOduration=5.084209186 podStartE2EDuration="23.001021131s" totalImagesPullingTime="17.916811945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:32.000434725 +0000 UTC m=+22.728251771" watchObservedRunningTime="2026-09-08 14:34:32.001021131 +0000 UTC m=+22.728838144"
Sep 08 14:34:32.718154 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:32.718046    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T14:34:31.98127633Z","UUID":"a1aff569-d73e-45be-8b3f-dc57bd7c0d05","Handler":null,"Name":"","Endpoint":""}
Sep 08 14:34:32.719632 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:32.719607    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 14:34:32.719743 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:32.719645    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 14:34:32.900027 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:32.899765    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal" event={"ID":"15f573fb3bfde2f504806a4ebee87ad0","Type":"ContainerStarted","Data":"d702d4faefcfae3eca6e1fe889e4ce91e291f3926134ee183497b9e038119443"}
Sep 08 14:34:32.902804 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:32.902773    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2" event={"ID":"c874f8e7-a9b5-44dc-bdde-4b21dff86a47","Type":"ContainerStarted","Data":"801feabf449c76919f50d85d67687536ebabe7374d1e6aa39afa6376e82f2101"}
Sep 08 14:34:32.917905 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:32.917853    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-6.ec2.internal" podStartSLOduration=22.917820642 podStartE2EDuration="22.917820642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:34:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:32.917694189 +0000 UTC m=+23.645511202" watchObservedRunningTime="2026-09-08 14:34:32.917820642 +0000 UTC m=+23.645637657"
Sep 08 14:34:33.787805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:33.787772    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:33.788012 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:33.787781    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:33.788012 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:33.787928    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:33.788137 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:33.788004    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:33.788137 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:33.788056    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:33.788234 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:33.788159    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:33.907553 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:33.907515    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2" event={"ID":"c874f8e7-a9b5-44dc-bdde-4b21dff86a47","Type":"ContainerStarted","Data":"916db00c266ea96d61af10b32f728e62b9e31a8b030d8713ead5201081cb904e"}
Sep 08 14:34:33.911561 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:33.911529    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h" event={"ID":"55f744b4-526c-4826-8bc5-46ace246b9d7","Type":"ContainerStarted","Data":"779272cf8cab6bc15974597fa9367ecdb5cba2bb82fc540922b086ab0b1b25e4"}
Sep 08 14:34:33.928987 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:33.928941    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wt2" podStartSLOduration=4.866670983 podStartE2EDuration="24.928927236s" totalImagesPullingTime="20.062256253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:33.927585653 +0000 UTC m=+24.655402670" watchObservedRunningTime="2026-09-08 14:34:33.928927236 +0000 UTC m=+24.656744251"
Sep 08 14:34:34.496055 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:34.496021    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:34.595970 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:34.595935    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:34.631001 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:34.630966    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:34.631650 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:34.631624    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:35.787199 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:35.787121    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:35.787199 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:35.787161    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:35.787658 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:35.787125    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:35.787658 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:35.787251    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:35.787658 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:35.787334    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:35.787658 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:35.787410    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:36.013129 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.013091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:36.115053 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.114970    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:36.335932 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.335902    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:36.336092 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.336000    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 08 14:34:36.336437 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.336419    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fjchd"
Sep 08 14:34:36.442594 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.442563    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:36.920508 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.920278    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wbz8" event={"ID":"78adf8d2-113a-4251-a33f-66236876be03","Type":"ContainerStarted","Data":"3ec915fc64af8ed75a6ea0356e03ce911ea5caae0c97ed42b0786ab28819d844"}
Sep 08 14:34:36.923861 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.923820    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h" event={"ID":"55f744b4-526c-4826-8bc5-46ace246b9d7","Type":"ContainerStarted","Data":"5b464514bd4c28a76100b193d6f048cd3f7bbe08c8e912d4df73ba0a89024ed5"}
Sep 08 14:34:36.924155 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.924126    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:36.924226 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.924206    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:36.939924 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.939872    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:36.972765 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:36.972728    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h" podStartSLOduration=9.312164574 podStartE2EDuration="27.972715515s" totalImagesPullingTime="18.660550941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:36.972364926 +0000 UTC m=+27.700181939" watchObservedRunningTime="2026-09-08 14:34:36.972715515 +0000 UTC m=+27.700532529"
Sep 08 14:34:37.787086 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:37.787050    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:37.787282 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:37.787093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:37.787282 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:37.787050    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:37.787282 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:37.787207    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:37.787449 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:37.787293    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:37.787449 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:37.787386    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:37.927602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:37.927568    2583 generic.go:356] "Generic (PLEG): container finished" podID="78adf8d2-113a-4251-a33f-66236876be03" containerID="3ec915fc64af8ed75a6ea0356e03ce911ea5caae0c97ed42b0786ab28819d844" exitCode=0
Sep 08 14:34:37.928129 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:37.927647    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wbz8" event={"ID":"78adf8d2-113a-4251-a33f-66236876be03","Type":"ContainerDied","Data":"3ec915fc64af8ed75a6ea0356e03ce911ea5caae0c97ed42b0786ab28819d844"}
Sep 08 14:34:37.928233 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:37.928214    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:37.943808 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:37.943783    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:34:38.162657 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:38.162627    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:38.262635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:38.262608    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:38.359504 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:38.359477    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ntzrt"]
Sep 08 14:34:38.359629 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:38.359612    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:38.359702 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:38.359688    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:38.362254 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:38.362195    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-php6q"]
Sep 08 14:34:38.362368 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:38.362277    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:38.362368 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:38.362339    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:38.378863 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:38.378820    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tmqvh"]
Sep 08 14:34:38.378965 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:38.378927    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:38.379016 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:38.378997    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:38.877085 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:38.877046    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:39.009201 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:39.009170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:39.788942 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:39.788732    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:39.789136 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:39.788792    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:39.789136 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:39.788807    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:39.789136 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:39.789108    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:39.789792 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:39.789456    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:39.789792 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:39.789564    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:39.935067 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:39.935034    2583 generic.go:356] "Generic (PLEG): container finished" podID="78adf8d2-113a-4251-a33f-66236876be03" containerID="24f65e33ac4ef76d2749d06172efc3b7cbc0d9cfeab1e87e6f3016ae590abada" exitCode=0
Sep 08 14:34:39.935242 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:39.935123    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wbz8" event={"ID":"78adf8d2-113a-4251-a33f-66236876be03","Type":"ContainerDied","Data":"24f65e33ac4ef76d2749d06172efc3b7cbc0d9cfeab1e87e6f3016ae590abada"}
Sep 08 14:34:39.976107 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:39.976055    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:40.100244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:40.100154    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:40.820981 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:40.820918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:40.887091 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:40.887066    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:40.939786 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:40.939761    2583 generic.go:356] "Generic (PLEG): container finished" podID="78adf8d2-113a-4251-a33f-66236876be03" containerID="984e4d9f9f9790117b367b70aee703a7b18f8b4ea8cd5648bf6cc36bc6bf6237" exitCode=0
Sep 08 14:34:40.939911 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:40.939801    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wbz8" event={"ID":"78adf8d2-113a-4251-a33f-66236876be03","Type":"ContainerDied","Data":"984e4d9f9f9790117b367b70aee703a7b18f8b4ea8cd5648bf6cc36bc6bf6237"}
Sep 08 14:34:41.787376 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:41.787299    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:41.787990 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:41.787426    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:41.787990 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:41.787508    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tmqvh" podUID="db5c60b0-c15e-496d-b2a0-8c76704ee8ae"
Sep 08 14:34:41.787990 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:41.787416    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-php6q" podUID="ac6d150a-3459-4e57-93c5-2c52ec02c6a3"
Sep 08 14:34:41.787990 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:41.787541    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:41.787990 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:41.787672    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzrt" podUID="22348c24-e76f-4cd1-8646-7f3338b45ec7"
Sep 08 14:34:43.417418 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.417382    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:43.417824 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:43.417560    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:43.417824 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:43.417654    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret podName:22348c24-e76f-4cd1-8646-7f3338b45ec7 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:15.41763283 +0000 UTC m=+66.145449841 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret") pod "global-pull-secret-syncer-ntzrt" (UID: "22348c24-e76f-4cd1-8646-7f3338b45ec7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 14:34:43.518400 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.518358    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grbd\" (UniqueName: \"kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd\") pod \"network-check-target-php6q\" (UID: \"ac6d150a-3459-4e57-93c5-2c52ec02c6a3\") " pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:43.518563 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.518437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:43.518563 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:43.518523    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 14:34:43.518563 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:43.518548    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 14:34:43.518563 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:43.518560    2583 projected.go:196] Error preparing data for projected volume kube-api-access-8grbd for pod openshift-network-diagnostics/network-check-target-php6q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:43.518711 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:43.518575    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:43.518711 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:43.518619    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd podName:ac6d150a-3459-4e57-93c5-2c52ec02c6a3 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:15.518604446 +0000 UTC m=+66.246421438 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8grbd" (UniqueName: "kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd") pod "network-check-target-php6q" (UID: "ac6d150a-3459-4e57-93c5-2c52ec02c6a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 14:34:43.518711 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:43.518634    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs podName:db5c60b0-c15e-496d-b2a0-8c76704ee8ae nodeName:}" failed. No retries permitted until 2026-09-08 14:35:15.518627573 +0000 UTC m=+66.246444565 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs") pod "network-metrics-daemon-tmqvh" (UID: "db5c60b0-c15e-496d-b2a0-8c76704ee8ae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 14:34:43.526122 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.526102    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-6.ec2.internal" event="NodeReady"
Sep 08 14:34:43.526216 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.526201    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 14:34:43.577899 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.577380    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gm2hx"]
Sep 08 14:34:43.579590 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.579550    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jj45j"]
Sep 08 14:34:43.580393 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.579810    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.581256 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.581236    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jj45j"
Sep 08 14:34:43.582465 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.582446    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 14:34:43.582553 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.582470    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rsnws\""
Sep 08 14:34:43.582619 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.582597    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 14:34:43.584123 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.584099    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 14:34:43.584254 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.584231    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 14:34:43.584407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.584387    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xpbhp\""
Sep 08 14:34:43.584463 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.584435    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 14:34:43.590357 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.590340    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jj45j"]
Sep 08 14:34:43.593712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.593692    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gm2hx"]
Sep 08 14:34:43.718339 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.718253    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:43.719257 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.719233    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d9310f1-f184-4d94-a56b-bbceccbabe88-cert\") pod \"ingress-canary-jj45j\" (UID: \"6d9310f1-f184-4d94-a56b-bbceccbabe88\") " pod="openshift-ingress-canary/ingress-canary-jj45j"
Sep 08 14:34:43.719351 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.719284    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxvp4\" (UniqueName: \"kubernetes.io/projected/150e4cb2-f873-48a4-a41b-3dffe587ec1b-kube-api-access-xxvp4\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.719351 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.719341    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/150e4cb2-f873-48a4-a41b-3dffe587ec1b-config-volume\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.719429 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.719362    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.719502 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.719474    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/150e4cb2-f873-48a4-a41b-3dffe587ec1b-tmp-dir\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.719613 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.719511    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpsp9\" (UniqueName: \"kubernetes.io/projected/6d9310f1-f184-4d94-a56b-bbceccbabe88-kube-api-access-cpsp9\") pod \"ingress-canary-jj45j\" (UID: \"6d9310f1-f184-4d94-a56b-bbceccbabe88\") " pod="openshift-ingress-canary/ingress-canary-jj45j"
Sep 08 14:34:43.787492 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.787457    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:34:43.787650 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.787511    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:34:43.787650 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.787529    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:34:43.791565 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.791276    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 14:34:43.791565 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.791287    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 14:34:43.791565 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.791297    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 14:34:43.791565 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.791304    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 14:34:43.791565 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.791317    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-42kmv\""
Sep 08 14:34:43.791565 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.791342    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nvk2x\""
Sep 08 14:34:43.798037 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.798011    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:43.820105 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.820074    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d9310f1-f184-4d94-a56b-bbceccbabe88-cert\") pod \"ingress-canary-jj45j\" (UID: \"6d9310f1-f184-4d94-a56b-bbceccbabe88\") " pod="openshift-ingress-canary/ingress-canary-jj45j"
Sep 08 14:34:43.820203 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.820125    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxvp4\" (UniqueName: \"kubernetes.io/projected/150e4cb2-f873-48a4-a41b-3dffe587ec1b-kube-api-access-xxvp4\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.820203 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.820160    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/150e4cb2-f873-48a4-a41b-3dffe587ec1b-config-volume\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.820203 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.820191    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.820366 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.820249    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/150e4cb2-f873-48a4-a41b-3dffe587ec1b-tmp-dir\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.820366 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.820267    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpsp9\" (UniqueName: \"kubernetes.io/projected/6d9310f1-f184-4d94-a56b-bbceccbabe88-kube-api-access-cpsp9\") pod \"ingress-canary-jj45j\" (UID: \"6d9310f1-f184-4d94-a56b-bbceccbabe88\") " pod="openshift-ingress-canary/ingress-canary-jj45j"
Sep 08 14:34:43.820366 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:43.820348    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:34:43.820482 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:43.820416    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls podName:150e4cb2-f873-48a4-a41b-3dffe587ec1b nodeName:}" failed. No retries permitted until 2026-09-08 14:34:44.320395964 +0000 UTC m=+35.048212972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls") pod "dns-default-gm2hx" (UID: "150e4cb2-f873-48a4-a41b-3dffe587ec1b") : secret "dns-default-metrics-tls" not found
Sep 08 14:34:43.820618 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.820598    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/150e4cb2-f873-48a4-a41b-3dffe587ec1b-tmp-dir\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.820755 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.820736    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/150e4cb2-f873-48a4-a41b-3dffe587ec1b-config-volume\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.824763 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.824740    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d9310f1-f184-4d94-a56b-bbceccbabe88-cert\") pod \"ingress-canary-jj45j\" (UID: \"6d9310f1-f184-4d94-a56b-bbceccbabe88\") " pod="openshift-ingress-canary/ingress-canary-jj45j"
Sep 08 14:34:43.831679 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.831656    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxvp4\" (UniqueName: \"kubernetes.io/projected/150e4cb2-f873-48a4-a41b-3dffe587ec1b-kube-api-access-xxvp4\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:43.831802 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.831787    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpsp9\" (UniqueName: \"kubernetes.io/projected/6d9310f1-f184-4d94-a56b-bbceccbabe88-kube-api-access-cpsp9\") pod \"ingress-canary-jj45j\" (UID: \"6d9310f1-f184-4d94-a56b-bbceccbabe88\") " pod="openshift-ingress-canary/ingress-canary-jj45j"
Sep 08 14:34:43.901330 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:43.901300    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jj45j"
Sep 08 14:34:44.324177 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:44.324136    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:44.324353 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:44.324292    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:34:44.324406 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:44.324362    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls podName:150e4cb2-f873-48a4-a41b-3dffe587ec1b nodeName:}" failed. No retries permitted until 2026-09-08 14:34:45.32434143 +0000 UTC m=+36.052158444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls") pod "dns-default-gm2hx" (UID: "150e4cb2-f873-48a4-a41b-3dffe587ec1b") : secret "dns-default-metrics-tls" not found
Sep 08 14:34:45.031982 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:45.031939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:45.330540 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:45.330314    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:45.330639 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:45.330473    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:34:45.330680 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:45.330664    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls podName:150e4cb2-f873-48a4-a41b-3dffe587ec1b nodeName:}" failed. No retries permitted until 2026-09-08 14:34:47.330643376 +0000 UTC m=+38.058460389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls") pod "dns-default-gm2hx" (UID: "150e4cb2-f873-48a4-a41b-3dffe587ec1b") : secret "dns-default-metrics-tls" not found
Sep 08 14:34:45.546014 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:45.545983    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jj45j"]
Sep 08 14:34:45.549280 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:45.549253    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9310f1_f184_4d94_a56b_bbceccbabe88.slice/crio-735ed32ee73cbedaae5392d598ee92a5be907707c5e01fdea1be579ef53f9d25 WatchSource:0}: Error finding container 735ed32ee73cbedaae5392d598ee92a5be907707c5e01fdea1be579ef53f9d25: Status 404 returned error can't find the container with id 735ed32ee73cbedaae5392d598ee92a5be907707c5e01fdea1be579ef53f9d25
Sep 08 14:34:45.952964 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:45.952881    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jj45j" event={"ID":"6d9310f1-f184-4d94-a56b-bbceccbabe88","Type":"ContainerStarted","Data":"735ed32ee73cbedaae5392d598ee92a5be907707c5e01fdea1be579ef53f9d25"}
Sep 08 14:34:45.955480 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:45.955456    2583 generic.go:356] "Generic (PLEG): container finished" podID="78adf8d2-113a-4251-a33f-66236876be03" containerID="346b98b0aaa06d715d9a6b697fd2904ac923edac28d64fe409bec448f56eda0a" exitCode=0
Sep 08 14:34:45.955602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:45.955488    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wbz8" event={"ID":"78adf8d2-113a-4251-a33f-66236876be03","Type":"ContainerDied","Data":"346b98b0aaa06d715d9a6b697fd2904ac923edac28d64fe409bec448f56eda0a"}
Sep 08 14:34:46.961671 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:46.961639    2583 generic.go:356] "Generic (PLEG): container finished" podID="78adf8d2-113a-4251-a33f-66236876be03" containerID="460ba524f1d7ef2ab2692160d64eef37cf16a97522535c608f332bbb26f2bf4d" exitCode=0
Sep 08 14:34:46.962351 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:46.961712    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wbz8" event={"ID":"78adf8d2-113a-4251-a33f-66236876be03","Type":"ContainerDied","Data":"460ba524f1d7ef2ab2692160d64eef37cf16a97522535c608f332bbb26f2bf4d"}
Sep 08 14:34:47.346789 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:47.346756    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:47.346985 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:47.346950    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:34:47.347053 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:47.347030    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls podName:150e4cb2-f873-48a4-a41b-3dffe587ec1b nodeName:}" failed. No retries permitted until 2026-09-08 14:34:51.347012467 +0000 UTC m=+42.074829463 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls") pod "dns-default-gm2hx" (UID: "150e4cb2-f873-48a4-a41b-3dffe587ec1b") : secret "dns-default-metrics-tls" not found
Sep 08 14:34:47.967864 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:47.967818    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wbz8" event={"ID":"78adf8d2-113a-4251-a33f-66236876be03","Type":"ContainerStarted","Data":"6c37f6923d96a2896c580b33b192b7fd84b5facf40c6f915adfe250add41d80e"}
Sep 08 14:34:47.994069 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:47.994018    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5wbz8" podStartSLOduration=19.641487781 podStartE2EDuration="38.994001653s" totalImagesPullingTime="19.352513872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:47.993310376 +0000 UTC m=+38.721127390" watchObservedRunningTime="2026-09-08 14:34:47.994001653 +0000 UTC m=+38.721818666"
Sep 08 14:34:48.118082 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:48.118053    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:48.183545 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:48.183473    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:48.369510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:48.369475    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:48.674301 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:48.674112    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:48.971293 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:48.971255    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jj45j" event={"ID":"6d9310f1-f184-4d94-a56b-bbceccbabe88","Type":"ContainerStarted","Data":"dc22781bc4b01f64beee5d2bd317a70474feb384376c7876feae32ba49ca6dbb"}
Sep 08 14:34:51.377797 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:51.377760    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:51.378180 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:51.377927    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:34:51.378180 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:51.377990    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls podName:150e4cb2-f873-48a4-a41b-3dffe587ec1b nodeName:}" failed. No retries permitted until 2026-09-08 14:34:59.377975658 +0000 UTC m=+50.105792665 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls") pod "dns-default-gm2hx" (UID: "150e4cb2-f873-48a4-a41b-3dffe587ec1b") : secret "dns-default-metrics-tls" not found
Sep 08 14:34:51.479245 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:51.479206    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:51.575751 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:51.575720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:54.830559 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:54.830518    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:54.902575 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:54.902542    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:55.818956 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:55.818907    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jj45j" podStartSLOduration=10.044908783 podStartE2EDuration="12.818890856s" totalImagesPullingTime="2.773982073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:34:48.996962355 +0000 UTC m=+39.724779369" watchObservedRunningTime="2026-09-08 14:34:55.818890856 +0000 UTC m=+46.546707870"
Sep 08 14:34:55.819112 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:55.819053    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mj4rs"]
Sep 08 14:34:55.851211 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:55.851180    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mj4rs"]
Sep 08 14:34:55.851571 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:55.851285    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:34:55.854178 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:55.854155    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 08 14:34:55.854397 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:55.854377    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 08 14:34:55.854486 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:55.854400    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4wjwr\""
Sep 08 14:34:55.936895 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:55.936863    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-4hz9j"]
Sep 08 14:34:55.958155 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:55.958121    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-4hz9j"
Sep 08 14:34:55.959070 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:55.959045    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-4hz9j"]
Sep 08 14:34:55.960988 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:55.960967    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-hxbsf\""
Sep 08 14:34:56.011177 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.011149    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:34:56.011329 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.011211    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3094d0b1-9ac4-42d5-824a-0fe574f90432-nginx-conf\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:34:56.023256 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.023226    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"]
Sep 08 14:34:56.044912 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.044884    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"]
Sep 08 14:34:56.045029 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.044996    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:34:56.048047 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.048027    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9rzfw\""
Sep 08 14:34:56.048194 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.048173    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 08 14:34:56.048520 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.048504    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 08 14:34:56.048969 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.048954    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 08 14:34:56.112470 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.112408    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:34:56.112582 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.112471    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3094d0b1-9ac4-42d5-824a-0fe574f90432-nginx-conf\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:34:56.112582 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.112510    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75k2t\" (UniqueName: \"kubernetes.io/projected/8b3d3ba2-74ef-481b-b8bc-f260ef1fd98a-kube-api-access-75k2t\") pod \"network-check-source-685456655f-4hz9j\" (UID: \"8b3d3ba2-74ef-481b-b8bc-f260ef1fd98a\") " pod="openshift-network-diagnostics/network-check-source-685456655f-4hz9j"
Sep 08 14:34:56.112582 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.112542    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 14:34:56.112696 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.112612    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert podName:3094d0b1-9ac4-42d5-824a-0fe574f90432 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:56.61259578 +0000 UTC m=+47.340412797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mj4rs" (UID: "3094d0b1-9ac4-42d5-824a-0fe574f90432") : secret "networking-console-plugin-cert" not found
Sep 08 14:34:56.113180 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.113163    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3094d0b1-9ac4-42d5-824a-0fe574f90432-nginx-conf\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:34:56.128316 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.128291    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"]
Sep 08 14:34:56.149744 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.149726    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-65fd4f6866-nr2wr"]
Sep 08 14:34:56.149894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.149878    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:56.154659 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.154641    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 14:34:56.155494 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.155476    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 14:34:56.157094 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.156855    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 08 14:34:56.157094 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.156884    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 08 14:34:56.157450 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.157424    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kd4kk\""
Sep 08 14:34:56.169409 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.169390    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"]
Sep 08 14:34:56.169535 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.169419    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65fd4f6866-nr2wr"]
Sep 08 14:34:56.169535 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.169505    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.172113 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.172095    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 08 14:34:56.172406 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.172382    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 08 14:34:56.172406 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.172397    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 08 14:34:56.172547 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.172466    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 08 14:34:56.172743 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.172723    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 14:34:56.172862 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.172742    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6ggrd\""
Sep 08 14:34:56.173717 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.173699    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 14:34:56.213614 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.213589    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn7b9\" (UniqueName: \"kubernetes.io/projected/7ed7677e-b23c-494a-a657-dc57533997b6-kube-api-access-gn7b9\") pod \"cluster-samples-operator-7cdcfcf76c-rzrqx\" (UID: \"7ed7677e-b23c-494a-a657-dc57533997b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:34:56.213722 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.213628    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75k2t\" (UniqueName: \"kubernetes.io/projected/8b3d3ba2-74ef-481b-b8bc-f260ef1fd98a-kube-api-access-75k2t\") pod \"network-check-source-685456655f-4hz9j\" (UID: \"8b3d3ba2-74ef-481b-b8bc-f260ef1fd98a\") " pod="openshift-network-diagnostics/network-check-source-685456655f-4hz9j"
Sep 08 14:34:56.213722 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.213706    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rzrqx\" (UID: \"7ed7677e-b23c-494a-a657-dc57533997b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:34:56.223042 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.223018    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75k2t\" (UniqueName: \"kubernetes.io/projected/8b3d3ba2-74ef-481b-b8bc-f260ef1fd98a-kube-api-access-75k2t\") pod \"network-check-source-685456655f-4hz9j\" (UID: \"8b3d3ba2-74ef-481b-b8bc-f260ef1fd98a\") " pod="openshift-network-diagnostics/network-check-source-685456655f-4hz9j"
Sep 08 14:34:56.267796 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.267768    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-4hz9j"
Sep 08 14:34:56.279365 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.279341    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:56.314087 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.314062    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rzrqx\" (UID: \"7ed7677e-b23c-494a-a657-dc57533997b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:34:56.314223 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.314112    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:56.314223 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.314136    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqz8g\" (UniqueName: \"kubernetes.io/projected/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-kube-api-access-wqz8g\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.314223 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.314187    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-stats-auth\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.314223 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.314213    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-default-certificate\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.314373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.314232    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.314373 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.314244    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 14:34:56.314373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.314256    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f371cfd8-3a5b-45d1-ad8b-04d454e09460-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:56.314373 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.314310    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls podName:7ed7677e-b23c-494a-a657-dc57533997b6 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:56.814290983 +0000 UTC m=+47.542107993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rzrqx" (UID: "7ed7677e-b23c-494a-a657-dc57533997b6") : secret "samples-operator-tls" not found
Sep 08 14:34:56.314373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.314363    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn7b9\" (UniqueName: \"kubernetes.io/projected/7ed7677e-b23c-494a-a657-dc57533997b6-kube-api-access-gn7b9\") pod \"cluster-samples-operator-7cdcfcf76c-rzrqx\" (UID: \"7ed7677e-b23c-494a-a657-dc57533997b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:34:56.314546 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.314394    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.314546 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.314453    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhdt4\" (UniqueName: \"kubernetes.io/projected/f371cfd8-3a5b-45d1-ad8b-04d454e09460-kube-api-access-bhdt4\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:56.325064 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.325031    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn7b9\" (UniqueName: \"kubernetes.io/projected/7ed7677e-b23c-494a-a657-dc57533997b6-kube-api-access-gn7b9\") pod \"cluster-samples-operator-7cdcfcf76c-rzrqx\" (UID: \"7ed7677e-b23c-494a-a657-dc57533997b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:34:56.378425 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.378354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:56.414750 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.414726    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:56.414877 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.414757    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqz8g\" (UniqueName: \"kubernetes.io/projected/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-kube-api-access-wqz8g\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.414877 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.414776    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-stats-auth\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.414877 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.414797    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-default-certificate\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.414877 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.414817    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.414877 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.414862    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f371cfd8-3a5b-45d1-ad8b-04d454e09460-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:56.415127 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.414894    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.415127 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.414902    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 14:34:56.415127 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.414949    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:34:56.415127 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.414979    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls podName:f371cfd8-3a5b-45d1-ad8b-04d454e09460 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:56.914961111 +0000 UTC m=+47.642778105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzg2x" (UID: "f371cfd8-3a5b-45d1-ad8b-04d454e09460") : secret "cluster-monitoring-operator-tls" not found
Sep 08 14:34:56.415127 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.415008    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:56.914989988 +0000 UTC m=+47.642806997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : secret "router-metrics-certs-default" not found
Sep 08 14:34:56.415127 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.414953    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhdt4\" (UniqueName: \"kubernetes.io/projected/f371cfd8-3a5b-45d1-ad8b-04d454e09460-kube-api-access-bhdt4\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:56.415127 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.415062    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:56.915049817 +0000 UTC m=+47.642866809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : configmap references non-existent config key: service-ca.crt
Sep 08 14:34:56.415607 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.415587    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f371cfd8-3a5b-45d1-ad8b-04d454e09460-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:56.417426 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.417397    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-stats-auth\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.417524 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.417410    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-default-certificate\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.423637 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.423617    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhdt4\" (UniqueName: \"kubernetes.io/projected/f371cfd8-3a5b-45d1-ad8b-04d454e09460-kube-api-access-bhdt4\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:56.424117 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.424102    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqz8g\" (UniqueName: \"kubernetes.io/projected/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-kube-api-access-wqz8g\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.551740 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.551710    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-4hz9j"]
Sep 08 14:34:56.554729 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:34:56.554696    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b3d3ba2_74ef_481b_b8bc_f260ef1fd98a.slice/crio-07edc1ab55a82710d76d4035925c14eb1f83824d04cebefdecfdf0865ade8edd WatchSource:0}: Error finding container 07edc1ab55a82710d76d4035925c14eb1f83824d04cebefdecfdf0865ade8edd: Status 404 returned error can't find the container with id 07edc1ab55a82710d76d4035925c14eb1f83824d04cebefdecfdf0865ade8edd
Sep 08 14:34:56.616719 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.616687    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:34:56.616889 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.616865    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 14:34:56.616962 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.616952    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert podName:3094d0b1-9ac4-42d5-824a-0fe574f90432 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:57.616936032 +0000 UTC m=+48.344753025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mj4rs" (UID: "3094d0b1-9ac4-42d5-824a-0fe574f90432") : secret "networking-console-plugin-cert" not found
Sep 08 14:34:56.818813 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.818777    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rzrqx\" (UID: \"7ed7677e-b23c-494a-a657-dc57533997b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:34:56.818984 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.818925    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 14:34:56.819030 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.818990    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls podName:7ed7677e-b23c-494a-a657-dc57533997b6 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:57.818974245 +0000 UTC m=+48.546791237 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rzrqx" (UID: "7ed7677e-b23c-494a-a657-dc57533997b6") : secret "samples-operator-tls" not found
Sep 08 14:34:56.919736 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.919701    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.920119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.919803    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:56.920119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.919864    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:56.920119 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.919905    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:57.919882497 +0000 UTC m=+48.647699499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : configmap references non-existent config key: service-ca.crt
Sep 08 14:34:56.920119 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.919922    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 14:34:56.920119 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.919955    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:34:56.920119 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.919969    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls podName:f371cfd8-3a5b-45d1-ad8b-04d454e09460 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:57.919957551 +0000 UTC m=+48.647774569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzg2x" (UID: "f371cfd8-3a5b-45d1-ad8b-04d454e09460") : secret "cluster-monitoring-operator-tls" not found
Sep 08 14:34:56.920119 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:56.919995    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:57.919983312 +0000 UTC m=+48.647800303 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : secret "router-metrics-certs-default" not found
Sep 08 14:34:56.989087 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:56.989055    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-4hz9j" event={"ID":"8b3d3ba2-74ef-481b-b8bc-f260ef1fd98a","Type":"ContainerStarted","Data":"07edc1ab55a82710d76d4035925c14eb1f83824d04cebefdecfdf0865ade8edd"}
Sep 08 14:34:57.625280 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:57.625247    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:34:57.625432 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:57.625402    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 14:34:57.625473 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:57.625467    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert podName:3094d0b1-9ac4-42d5-824a-0fe574f90432 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:59.625451717 +0000 UTC m=+50.353268729 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mj4rs" (UID: "3094d0b1-9ac4-42d5-824a-0fe574f90432") : secret "networking-console-plugin-cert" not found
Sep 08 14:34:57.826891 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:57.826857    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rzrqx\" (UID: \"7ed7677e-b23c-494a-a657-dc57533997b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:34:57.827053 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:57.827000    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 14:34:57.827096 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:57.827061    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls podName:7ed7677e-b23c-494a-a657-dc57533997b6 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:59.827042211 +0000 UTC m=+50.554859221 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rzrqx" (UID: "7ed7677e-b23c-494a-a657-dc57533997b6") : secret "samples-operator-tls" not found
Sep 08 14:34:57.883212 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:57.883141    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:57.927585 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:57.927550    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:57.927984 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:57.927664    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:57.927984 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:57.927708    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:59.927690299 +0000 UTC m=+50.655507304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : configmap references non-existent config key: service-ca.crt
Sep 08 14:34:57.927984 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:57.927733    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:57.927984 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:57.927778    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 14:34:57.927984 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:57.927815    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:34:57.927984 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:57.927854    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls podName:f371cfd8-3a5b-45d1-ad8b-04d454e09460 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:59.92781295 +0000 UTC m=+50.655629947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzg2x" (UID: "f371cfd8-3a5b-45d1-ad8b-04d454e09460") : secret "cluster-monitoring-operator-tls" not found
Sep 08 14:34:57.927984 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:57.927874    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:34:59.927863647 +0000 UTC m=+50.655680659 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : secret "router-metrics-certs-default" not found
Sep 08 14:34:57.980800 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:57.980747    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:59.443171 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:59.443137    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:34:59.443709 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:59.443303    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 14:34:59.443709 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:59.443381    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls podName:150e4cb2-f873-48a4-a41b-3dffe587ec1b nodeName:}" failed. No retries permitted until 2026-09-08 14:35:15.443357224 +0000 UTC m=+66.171174217 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls") pod "dns-default-gm2hx" (UID: "150e4cb2-f873-48a4-a41b-3dffe587ec1b") : secret "dns-default-metrics-tls" not found
Sep 08 14:34:59.645368 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:59.645329    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:34:59.645538 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:59.645491    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 14:34:59.645597 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:59.645556    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert podName:3094d0b1-9ac4-42d5-824a-0fe574f90432 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:03.645540622 +0000 UTC m=+54.373357613 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mj4rs" (UID: "3094d0b1-9ac4-42d5-824a-0fe574f90432") : secret "networking-console-plugin-cert" not found
Sep 08 14:34:59.828040 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:59.828003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:59.847270 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:59.847242    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rzrqx\" (UID: \"7ed7677e-b23c-494a-a657-dc57533997b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:34:59.847438 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:59.847411    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 14:34:59.847499 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:59.847486    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls podName:7ed7677e-b23c-494a-a657-dc57533997b6 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:03.847468747 +0000 UTC m=+54.575285762 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rzrqx" (UID: "7ed7677e-b23c-494a-a657-dc57533997b6") : secret "samples-operator-tls" not found
Sep 08 14:34:59.928419 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:59.928381    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:34:59.948555 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:59.948525    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:59.948739 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:59.948651    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:34:59.948739 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:59.948684    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:03.948659654 +0000 UTC m=+54.676476662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : configmap references non-existent config key: service-ca.crt
Sep 08 14:34:59.948933 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:59.948743    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 14:34:59.948933 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:34:59.948755    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:34:59.948933 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:59.948796    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls podName:f371cfd8-3a5b-45d1-ad8b-04d454e09460 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:03.948778571 +0000 UTC m=+54.676595574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzg2x" (UID: "f371cfd8-3a5b-45d1-ad8b-04d454e09460") : secret "cluster-monitoring-operator-tls" not found
Sep 08 14:34:59.948933 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:59.948883    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:34:59.948933 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:34:59.948923    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:03.948910394 +0000 UTC m=+54.676727392 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : secret "router-metrics-certs-default" not found
Sep 08 14:35:01.001187 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.001146    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-4hz9j" event={"ID":"8b3d3ba2-74ef-481b-b8bc-f260ef1fd98a","Type":"ContainerStarted","Data":"930a3a4919d013cc71578e72752e643b8098ea0c6d5c212606d997ffd3404d91"}
Sep 08 14:35:01.017857 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.017786    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-4hz9j" podStartSLOduration=1.994140356 podStartE2EDuration="6.017770223s" totalImagesPullingTime="4.023629867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:01.017606894 +0000 UTC m=+51.745423912" watchObservedRunningTime="2026-09-08 14:35:01.017770223 +0000 UTC m=+51.745587238"
Sep 08 14:35:01.532106 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.532074    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nzcsk"]
Sep 08 14:35:01.534019 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.533998    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.535115 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.535090    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:01.538540 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.538523    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7h2nw\""
Sep 08 14:35:01.538859 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.538842    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 14:35:01.538953 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.538874    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 14:35:01.539539 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.539517    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 14:35:01.539771 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.539756    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 14:35:01.551963 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.551947    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nzcsk"]
Sep 08 14:35:01.562264 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.562244    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a3c361d8-f0d5-45fb-b228-11c76fbec590-crio-socket\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.562349 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.562291    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.562400 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.562381    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3c361d8-f0d5-45fb-b228-11c76fbec590-data-volume\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.562434 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.562409    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmf6l\" (UniqueName: \"kubernetes.io/projected/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-api-access-xmf6l\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.562494 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.562480    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.605690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.605661    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:01.662768 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.662742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.662907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.662774    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a3c361d8-f0d5-45fb-b228-11c76fbec590-crio-socket\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.662907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.662808    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.662907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.662875    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3c361d8-f0d5-45fb-b228-11c76fbec590-data-volume\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.662907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.662892    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmf6l\" (UniqueName: \"kubernetes.io/projected/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-api-access-xmf6l\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.663060 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:01.662915    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:01.663060 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:01.662984    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls podName:a3c361d8-f0d5-45fb-b228-11c76fbec590 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:02.162962431 +0000 UTC m=+52.890779439 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzcsk" (UID: "a3c361d8-f0d5-45fb-b228-11c76fbec590") : secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:01.663060 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:01.663003    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 14:35:01.663060 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.663026    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a3c361d8-f0d5-45fb-b228-11c76fbec590-crio-socket\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.663201 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:01.663065    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm podName:a3c361d8-f0d5-45fb-b228-11c76fbec590 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:02.163047812 +0000 UTC m=+52.890864820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzcsk" (UID: "a3c361d8-f0d5-45fb-b228-11c76fbec590") : configmap "kube-rbac-proxy" not found
Sep 08 14:35:01.663240 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.663230    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3c361d8-f0d5-45fb-b228-11c76fbec590-data-volume\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.677195 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.677162    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmf6l\" (UniqueName: \"kubernetes.io/projected/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-api-access-xmf6l\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:01.738597 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:01.738558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:02.063058 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:02.063026    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:02.157372 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:02.157312    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:02.166548 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:02.166520    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:02.166676 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:02.166577    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:02.166676 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:02.166663    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:02.166748 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:02.166701    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 14:35:02.166748 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:02.166723    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls podName:a3c361d8-f0d5-45fb-b228-11c76fbec590 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:03.166707952 +0000 UTC m=+53.894524957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzcsk" (UID: "a3c361d8-f0d5-45fb-b228-11c76fbec590") : secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:02.166748 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:02.166746    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm podName:a3c361d8-f0d5-45fb-b228-11c76fbec590 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:03.166734551 +0000 UTC m=+53.894551543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzcsk" (UID: "a3c361d8-f0d5-45fb-b228-11c76fbec590") : configmap "kube-rbac-proxy" not found
Sep 08 14:35:02.426769 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:02.426696    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:02.524684 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:02.524655    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:03.174691 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:03.174652    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:03.175115 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:03.174710    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:03.175115 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.174814    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:03.175115 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.174886    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 14:35:03.175115 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.174921    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls podName:a3c361d8-f0d5-45fb-b228-11c76fbec590 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:05.174902058 +0000 UTC m=+55.902719067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzcsk" (UID: "a3c361d8-f0d5-45fb-b228-11c76fbec590") : secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:03.175115 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.174942    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm podName:a3c361d8-f0d5-45fb-b228-11c76fbec590 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:05.17492878 +0000 UTC m=+55.902745772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzcsk" (UID: "a3c361d8-f0d5-45fb-b228-11c76fbec590") : configmap "kube-rbac-proxy" not found
Sep 08 14:35:03.645645 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:03.645618    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vsxtr_d88ea008-9ce7-4a83-abd8-bbde485345a3/dns-node-resolver/0.log"
Sep 08 14:35:03.679245 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:03.679211    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:35:03.679402 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.679382    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 14:35:03.679476 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.679467    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert podName:3094d0b1-9ac4-42d5-824a-0fe574f90432 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:11.679445597 +0000 UTC m=+62.407262609 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mj4rs" (UID: "3094d0b1-9ac4-42d5-824a-0fe574f90432") : secret "networking-console-plugin-cert" not found
Sep 08 14:35:03.881076 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:03.881036    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rzrqx\" (UID: \"7ed7677e-b23c-494a-a657-dc57533997b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:35:03.881250 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.881174    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 14:35:03.881250 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.881233    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls podName:7ed7677e-b23c-494a-a657-dc57533997b6 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:11.881217732 +0000 UTC m=+62.609034723 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rzrqx" (UID: "7ed7677e-b23c-494a-a657-dc57533997b6") : secret "samples-operator-tls" not found
Sep 08 14:35:03.982326 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:03.982283    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:03.982479 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:03.982342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:03.982479 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.982437    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:11.982421886 +0000 UTC m=+62.710238878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : configmap references non-existent config key: service-ca.crt
Sep 08 14:35:03.982555 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.982473    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:35:03.982555 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:03.982482    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:35:03.982555 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.982525    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:11.982510869 +0000 UTC m=+62.710327884 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : secret "router-metrics-certs-default" not found
Sep 08 14:35:03.982648 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.982569    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 14:35:03.982648 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:03.982606    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls podName:f371cfd8-3a5b-45d1-ad8b-04d454e09460 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:11.982598993 +0000 UTC m=+62.710415984 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzg2x" (UID: "f371cfd8-3a5b-45d1-ad8b-04d454e09460") : secret "cluster-monitoring-operator-tls" not found
Sep 08 14:35:04.052742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:04.052705    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:04.220229 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:04.220181    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:04.646618 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:04.646593    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hvzkl_2a15e5b5-3ba1-4587-943a-a4989e67b57b/node-ca/0.log"
Sep 08 14:35:05.045053 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:05.045031    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jj45j_6d9310f1-f184-4d94-a56b-bbceccbabe88/serve-healthcheck-canary/0.log"
Sep 08 14:35:05.191315 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:05.191275    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:05.191489 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:05.191399    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 14:35:05.191489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:05.191433    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:05.191489 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:05.191462    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm podName:a3c361d8-f0d5-45fb-b228-11c76fbec590 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:09.1914462 +0000 UTC m=+59.919263198 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzcsk" (UID: "a3c361d8-f0d5-45fb-b228-11c76fbec590") : configmap "kube-rbac-proxy" not found
Sep 08 14:35:05.191667 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:05.191543    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:05.191667 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:05.191604    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls podName:a3c361d8-f0d5-45fb-b228-11c76fbec590 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:09.191587887 +0000 UTC m=+59.919404900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzcsk" (UID: "a3c361d8-f0d5-45fb-b228-11c76fbec590") : secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:05.475682 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:05.475654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:07.819565 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:07.819533    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:08.425979 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:08.425942    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:08.637815 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:08.637782    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:09.223529 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:09.223487    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:09.224018 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:09.223617    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 14:35:09.224018 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:09.223657    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:09.224018 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:09.223679    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm podName:a3c361d8-f0d5-45fb-b228-11c76fbec590 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:17.223663309 +0000 UTC m=+67.951480316 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzcsk" (UID: "a3c361d8-f0d5-45fb-b228-11c76fbec590") : configmap "kube-rbac-proxy" not found
Sep 08 14:35:09.224018 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:09.223760    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:09.224018 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:09.223822    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls podName:a3c361d8-f0d5-45fb-b228-11c76fbec590 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:17.223806051 +0000 UTC m=+67.951623061 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzcsk" (UID: "a3c361d8-f0d5-45fb-b228-11c76fbec590") : secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:09.525093 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:09.525005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:09.701146 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:09.701114    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:09.888169 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:09.888085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:09.945848 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:09.945812    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lt7h"
Sep 08 14:35:10.088154 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:10.088126    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:10.296225 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:10.296182    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:10.704655 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:10.704549    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:11.312674 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:11.312644    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:11.487057 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:11.487028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:11.608093 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:11.608014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:11.709358 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:11.709329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:11.745122 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:11.745097    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:35:11.745269 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:11.745253    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 14:35:11.745339 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:11.745330    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert podName:3094d0b1-9ac4-42d5-824a-0fe574f90432 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:27.745313962 +0000 UTC m=+78.473130957 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mj4rs" (UID: "3094d0b1-9ac4-42d5-824a-0fe574f90432") : secret "networking-console-plugin-cert" not found
Sep 08 14:35:11.946887 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:11.946777    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rzrqx\" (UID: \"7ed7677e-b23c-494a-a657-dc57533997b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:35:11.949288 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:11.949257    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ed7677e-b23c-494a-a657-dc57533997b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rzrqx\" (UID: \"7ed7677e-b23c-494a-a657-dc57533997b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:35:11.957042 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:11.957022    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9rzfw\""
Sep 08 14:35:11.964673 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:11.964658    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"
Sep 08 14:35:12.047624 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:12.047593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:35:12.047757 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:12.047647    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:12.047757 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:12.047692    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:12.047757 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:12.047725    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 14:35:12.047757 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:12.047740    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 14:35:12.047929 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:12.047786    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls podName:f371cfd8-3a5b-45d1-ad8b-04d454e09460 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:28.047771737 +0000 UTC m=+78.775588733 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lzg2x" (UID: "f371cfd8-3a5b-45d1-ad8b-04d454e09460") : secret "cluster-monitoring-operator-tls" not found
Sep 08 14:35:12.047929 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:12.047801    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:28.047794838 +0000 UTC m=+78.775611831 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : secret "router-metrics-certs-default" not found
Sep 08 14:35:12.047929 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:12.047820    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle podName:fea5d1a3-561b-40e9-9978-9f1e662ef0c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:28.04780601 +0000 UTC m=+78.775623002 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle") pod "router-default-65fd4f6866-nr2wr" (UID: "fea5d1a3-561b-40e9-9978-9f1e662ef0c9") : configmap references non-existent config key: service-ca.crt
Sep 08 14:35:12.095638 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:12.095614    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:12.240270 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:12.240237    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx"]
Sep 08 14:35:12.289331 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:12.289307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:12.694481 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:12.694394    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:13.030629 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:13.030591    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx" event={"ID":"7ed7677e-b23c-494a-a657-dc57533997b6","Type":"ContainerStarted","Data":"7e315f3c3a527e0bb6c11b50e5d8ffd87123b507084f31ddd713dd48d76d9def"}
Sep 08 14:35:13.077232 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:13.077198    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:13.471991 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:13.471958    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:13.673252 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:13.673218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:14.281474 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:14.281395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:14.556695 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:14.556618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:14.688662 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:14.688628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:14.786103 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:14.786076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:14.886785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:14.886714    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:15.036548 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.036514    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx" event={"ID":"7ed7677e-b23c-494a-a657-dc57533997b6","Type":"ContainerStarted","Data":"279c200e5e541f40e4e5b561f1c2dfd902263dd94a9dc8123f241c76eb7e5c20"}
Sep 08 14:35:15.036709 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.036562    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx" event={"ID":"7ed7677e-b23c-494a-a657-dc57533997b6","Type":"ContainerStarted","Data":"dd8d1866183a0d4e904f0016fed997fd082e15f04d2d175eb4a8d93876ddf96f"}
Sep 08 14:35:15.055560 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.055515    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rzrqx" podStartSLOduration=17.322535199 podStartE2EDuration="19.055499871s" totalImagesPullingTime="1.732964672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:15.054988644 +0000 UTC m=+65.782805658" watchObservedRunningTime="2026-09-08 14:35:15.055499871 +0000 UTC m=+65.783316884"
Sep 08 14:35:15.476863 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.476815    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:35:15.477193 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.476888    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:35:15.479098 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.479080    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/150e4cb2-f873-48a4-a41b-3dffe587ec1b-metrics-tls\") pod \"dns-default-gm2hx\" (UID: \"150e4cb2-f873-48a4-a41b-3dffe587ec1b\") " pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:35:15.480462 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.480447    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 14:35:15.489000 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.488983    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22348c24-e76f-4cd1-8646-7f3338b45ec7-original-pull-secret\") pod \"global-pull-secret-syncer-ntzrt\" (UID: \"22348c24-e76f-4cd1-8646-7f3338b45ec7\") " pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:35:15.577374 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.577345    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:35:15.577498 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.577483    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grbd\" (UniqueName: \"kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd\") pod \"network-check-target-php6q\" (UID: \"ac6d150a-3459-4e57-93c5-2c52ec02c6a3\") " pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:35:15.579743 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.579726    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8grbd\" (UniqueName: \"kubernetes.io/projected/ac6d150a-3459-4e57-93c5-2c52ec02c6a3-kube-api-access-8grbd\") pod \"network-check-target-php6q\" (UID: \"ac6d150a-3459-4e57-93c5-2c52ec02c6a3\") " pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:35:15.580339 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.580320    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 14:35:15.589616 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.589593    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5c60b0-c15e-496d-b2a0-8c76704ee8ae-metrics-certs\") pod \"network-metrics-daemon-tmqvh\" (UID: \"db5c60b0-c15e-496d-b2a0-8c76704ee8ae\") " pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:35:15.601415 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.601394    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-42kmv\""
Sep 08 14:35:15.608555 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.608541    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tmqvh"
Sep 08 14:35:15.608605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.608576    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nvk2x\""
Sep 08 14:35:15.610044 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.610029    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzrt"
Sep 08 14:35:15.616712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.616634    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:35:15.696534 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.696510    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rsnws\""
Sep 08 14:35:15.703281 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.703261    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:35:15.876292 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.876259    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:15.883798 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.883775    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tmqvh"]
Sep 08 14:35:15.886726 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:15.886701    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5c60b0_c15e_496d_b2a0_8c76704ee8ae.slice/crio-8188ec5e60e487023b552e76749f77875f633f4f882529404ab45af8d7c8c7c4 WatchSource:0}: Error finding container 8188ec5e60e487023b552e76749f77875f633f4f882529404ab45af8d7c8c7c4: Status 404 returned error can't find the container with id 8188ec5e60e487023b552e76749f77875f633f4f882529404ab45af8d7c8c7c4
Sep 08 14:35:15.906347 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.906323    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ntzrt"]
Sep 08 14:35:15.908306 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.908257    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-php6q"]
Sep 08 14:35:15.910236 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:15.910216    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22348c24_e76f_4cd1_8646_7f3338b45ec7.slice/crio-7f923cd1fe3d3cb8aeaee2c78403ac0463265376d25f1ed2100c668f0876ff03 WatchSource:0}: Error finding container 7f923cd1fe3d3cb8aeaee2c78403ac0463265376d25f1ed2100c668f0876ff03: Status 404 returned error can't find the container with id 7f923cd1fe3d3cb8aeaee2c78403ac0463265376d25f1ed2100c668f0876ff03
Sep 08 14:35:15.910711 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:15.910647    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6d150a_3459_4e57_93c5_2c52ec02c6a3.slice/crio-eb53c6d97f5b347946a09bab7a5e72a3fc05730286dedec65f523b611bb58de7 WatchSource:0}: Error finding container eb53c6d97f5b347946a09bab7a5e72a3fc05730286dedec65f523b611bb58de7: Status 404 returned error can't find the container with id eb53c6d97f5b347946a09bab7a5e72a3fc05730286dedec65f523b611bb58de7
Sep 08 14:35:15.977107 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:15.977081    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gm2hx"]
Sep 08 14:35:15.979624 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:15.979597    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150e4cb2_f873_48a4_a41b_3dffe587ec1b.slice/crio-c14dd60b7daebb538ec296bfe5b535a3057d673862a6907772df6b2c862abfaf WatchSource:0}: Error finding container c14dd60b7daebb538ec296bfe5b535a3057d673862a6907772df6b2c862abfaf: Status 404 returned error can't find the container with id c14dd60b7daebb538ec296bfe5b535a3057d673862a6907772df6b2c862abfaf
Sep 08 14:35:16.040341 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:16.040275    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-php6q" event={"ID":"ac6d150a-3459-4e57-93c5-2c52ec02c6a3","Type":"ContainerStarted","Data":"f6fb634f9f40dcc4ec359b36110f9fd5d2eb4ee70a48ba1aaf2aeffca7bcfa03"}
Sep 08 14:35:16.040464 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:16.040415    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:35:16.040464 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:16.040456    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-php6q" event={"ID":"ac6d150a-3459-4e57-93c5-2c52ec02c6a3","Type":"ContainerStarted","Data":"eb53c6d97f5b347946a09bab7a5e72a3fc05730286dedec65f523b611bb58de7"}
Sep 08 14:35:16.041523 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:16.041502    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gm2hx" event={"ID":"150e4cb2-f873-48a4-a41b-3dffe587ec1b","Type":"ContainerStarted","Data":"c14dd60b7daebb538ec296bfe5b535a3057d673862a6907772df6b2c862abfaf"}
Sep 08 14:35:16.042546 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:16.042510    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tmqvh" event={"ID":"db5c60b0-c15e-496d-b2a0-8c76704ee8ae","Type":"ContainerStarted","Data":"8188ec5e60e487023b552e76749f77875f633f4f882529404ab45af8d7c8c7c4"}
Sep 08 14:35:16.043516 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:16.043499    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ntzrt" event={"ID":"22348c24-e76f-4cd1-8646-7f3338b45ec7","Type":"ContainerStarted","Data":"7f923cd1fe3d3cb8aeaee2c78403ac0463265376d25f1ed2100c668f0876ff03"}
Sep 08 14:35:16.060124 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:16.060082    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-php6q" podStartSLOduration=67.060069365 podStartE2EDuration="1m7.060069365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:16.059307509 +0000 UTC m=+66.787124524" watchObservedRunningTime="2026-09-08 14:35:16.060069365 +0000 UTC m=+66.787886431"
Sep 08 14:35:16.842027 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:16.841989    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:16.904169 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:16.904130    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:17.290746 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:17.290592    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:17.290746 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:17.290662    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:17.290746 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:17.290726    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:17.291006 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:17.290782    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls podName:a3c361d8-f0d5-45fb-b228-11c76fbec590 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:33.290764413 +0000 UTC m=+84.018581420 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzcsk" (UID: "a3c361d8-f0d5-45fb-b228-11c76fbec590") : secret "insights-runtime-extractor-tls" not found
Sep 08 14:35:17.291144 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:17.291124    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3c361d8-f0d5-45fb-b228-11c76fbec590-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:17.876331 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:17.876291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:18.054544 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:18.054458    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tmqvh" event={"ID":"db5c60b0-c15e-496d-b2a0-8c76704ee8ae","Type":"ContainerStarted","Data":"cc2eff215a35b4eb52323082b1c0502423b252e8146cc577aeed47c94cea2adf"}
Sep 08 14:35:18.054544 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:18.054517    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tmqvh" event={"ID":"db5c60b0-c15e-496d-b2a0-8c76704ee8ae","Type":"ContainerStarted","Data":"d5847c88cb21bb7123848b3d771b9782c9471b04dc4e80353a6d4ed83119ec85"}
Sep 08 14:35:18.074694 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:18.074637    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tmqvh" podStartSLOduration=67.894106182 podStartE2EDuration="1m9.074616668s" totalImagesPullingTime="1.180510486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:18.072440257 +0000 UTC m=+68.800257273" watchObservedRunningTime="2026-09-08 14:35:18.074616668 +0000 UTC m=+68.802433683"
Sep 08 14:35:18.082088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:18.082066    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:18.877661 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:18.877623    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:18.971099 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:18.971059    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:19.063943 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:19.063906    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gm2hx" event={"ID":"150e4cb2-f873-48a4-a41b-3dffe587ec1b","Type":"ContainerStarted","Data":"cf716115dc9980e1f3aaf3435d9d777f9e57197373675aa2f4d05f1f9dc473c9"}
Sep 08 14:35:19.064125 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:19.064006    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gm2hx" event={"ID":"150e4cb2-f873-48a4-a41b-3dffe587ec1b","Type":"ContainerStarted","Data":"91762628f47e1462718d97360e73ee0221091e69f041b3f9f393fe4a12480a8e"}
Sep 08 14:35:19.086551 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:19.086504    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gm2hx" podStartSLOduration=34.015245458 podStartE2EDuration="36.086488972s" totalImagesPullingTime="2.071243514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:19.085675207 +0000 UTC m=+69.813492223" watchObservedRunningTime="2026-09-08 14:35:19.086488972 +0000 UTC m=+69.814305987"
Sep 08 14:35:20.068379 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:20.068348    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:35:21.026263 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:21.026228    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:21.072340 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:21.072303    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ntzrt" event={"ID":"22348c24-e76f-4cd1-8646-7f3338b45ec7","Type":"ContainerStarted","Data":"3e1fd47372878cb63085c1b9ebc8b13e3bf3e49ae88bbee1360f7332fe490c87"}
Sep 08 14:35:21.089475 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:21.089432    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ntzrt" podStartSLOduration=67.443503997 podStartE2EDuration="1m12.089414784s" totalImagesPullingTime="4.645910787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:21.088736865 +0000 UTC m=+71.816553879" watchObservedRunningTime="2026-09-08 14:35:21.089414784 +0000 UTC m=+71.817231831"
Sep 08 14:35:21.110692 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:21.110656    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:25.337334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.337302    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh"]
Sep 08 14:35:25.339994 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.339977    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh"
Sep 08 14:35:25.345236 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.345212    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 08 14:35:25.345236 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.345225    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jb6kk\""
Sep 08 14:35:25.345412 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.345230    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 08 14:35:25.345412 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.345264    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 08 14:35:25.346119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.345966    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 08 14:35:25.369071 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.369047    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh"]
Sep 08 14:35:25.378067 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.378032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:25.464238 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.464199    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47910cdc-b677-429a-91b4-788d18d80892-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5879cdc854-x9vnh\" (UID: \"47910cdc-b677-429a-91b4-788d18d80892\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh"
Sep 08 14:35:25.464407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.464293    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcgnm\" (UniqueName: \"kubernetes.io/projected/47910cdc-b677-429a-91b4-788d18d80892-kube-api-access-tcgnm\") pod \"managed-serviceaccount-addon-agent-5879cdc854-x9vnh\" (UID: \"47910cdc-b677-429a-91b4-788d18d80892\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh"
Sep 08 14:35:25.499992 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.499963    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:25.565390 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.565359    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcgnm\" (UniqueName: \"kubernetes.io/projected/47910cdc-b677-429a-91b4-788d18d80892-kube-api-access-tcgnm\") pod \"managed-serviceaccount-addon-agent-5879cdc854-x9vnh\" (UID: \"47910cdc-b677-429a-91b4-788d18d80892\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh"
Sep 08 14:35:25.565524 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.565447    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47910cdc-b677-429a-91b4-788d18d80892-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5879cdc854-x9vnh\" (UID: \"47910cdc-b677-429a-91b4-788d18d80892\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh"
Sep 08 14:35:25.568082 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.568065    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47910cdc-b677-429a-91b4-788d18d80892-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5879cdc854-x9vnh\" (UID: \"47910cdc-b677-429a-91b4-788d18d80892\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh"
Sep 08 14:35:25.574191 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.574172    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcgnm\" (UniqueName: \"kubernetes.io/projected/47910cdc-b677-429a-91b4-788d18d80892-kube-api-access-tcgnm\") pod \"managed-serviceaccount-addon-agent-5879cdc854-x9vnh\" (UID: \"47910cdc-b677-429a-91b4-788d18d80892\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh"
Sep 08 14:35:25.595915 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.595816    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:25.659935 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.659918    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh"
Sep 08 14:35:25.697108 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.697081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:25.934371 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:25.934305    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh"]
Sep 08 14:35:25.938627 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:25.938603    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47910cdc_b677_429a_91b4_788d18d80892.slice/crio-d17ced381ab90b415b2ed50c8691272996725140b0705febf1280aab52a89dd1 WatchSource:0}: Error finding container d17ced381ab90b415b2ed50c8691272996725140b0705febf1280aab52a89dd1: Status 404 returned error can't find the container with id d17ced381ab90b415b2ed50c8691272996725140b0705febf1280aab52a89dd1
Sep 08 14:35:26.089136 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:26.089103    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh" event={"ID":"47910cdc-b677-429a-91b4-788d18d80892","Type":"ContainerStarted","Data":"d17ced381ab90b415b2ed50c8691272996725140b0705febf1280aab52a89dd1"}
Sep 08 14:35:27.784108 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:27.784070    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:35:27.786732 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:27.786692    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3094d0b1-9ac4-42d5-824a-0fe574f90432-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mj4rs\" (UID: \"3094d0b1-9ac4-42d5-824a-0fe574f90432\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:35:27.963431 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:27.963403    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4wjwr\""
Sep 08 14:35:27.971447 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:27.971422    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs"
Sep 08 14:35:28.088119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.088037    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:35:28.088119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.088096    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:28.088298 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.088134    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:28.088816 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.088779    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-service-ca-bundle\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:28.091191 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.091165    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea5d1a3-561b-40e9-9978-9f1e662ef0c9-metrics-certs\") pod \"router-default-65fd4f6866-nr2wr\" (UID: \"fea5d1a3-561b-40e9-9978-9f1e662ef0c9\") " pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:28.091288 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.091274    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f371cfd8-3a5b-45d1-ad8b-04d454e09460-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lzg2x\" (UID: \"f371cfd8-3a5b-45d1-ad8b-04d454e09460\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:35:28.263972 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.263941    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kd4kk\""
Sep 08 14:35:28.271695 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.271667    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"
Sep 08 14:35:28.280775 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.280749    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6ggrd\""
Sep 08 14:35:28.288823 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.288794    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:28.806237 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.806168    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x"]
Sep 08 14:35:28.808791 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:28.808760    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf371cfd8_3a5b_45d1_ad8b_04d454e09460.slice/crio-374c980ced249f993ef84abfa971ec735abe18951b07b4d53e1c2ffc61ef441f WatchSource:0}: Error finding container 374c980ced249f993ef84abfa971ec735abe18951b07b4d53e1c2ffc61ef441f: Status 404 returned error can't find the container with id 374c980ced249f993ef84abfa971ec735abe18951b07b4d53e1c2ffc61ef441f
Sep 08 14:35:28.816129 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.816109    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mj4rs"]
Sep 08 14:35:28.819599 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:28.819336    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3094d0b1_9ac4_42d5_824a_0fe574f90432.slice/crio-7180108fea10ac8fb3d3a5fcee466d7fdb38c066ad798c39ad3e862903e78188 WatchSource:0}: Error finding container 7180108fea10ac8fb3d3a5fcee466d7fdb38c066ad798c39ad3e862903e78188: Status 404 returned error can't find the container with id 7180108fea10ac8fb3d3a5fcee466d7fdb38c066ad798c39ad3e862903e78188
Sep 08 14:35:28.821706 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:28.821672    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65fd4f6866-nr2wr"]
Sep 08 14:35:29.100729 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:29.100622    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x" event={"ID":"f371cfd8-3a5b-45d1-ad8b-04d454e09460","Type":"ContainerStarted","Data":"374c980ced249f993ef84abfa971ec735abe18951b07b4d53e1c2ffc61ef441f"}
Sep 08 14:35:29.102217 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:29.102186    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh" event={"ID":"47910cdc-b677-429a-91b4-788d18d80892","Type":"ContainerStarted","Data":"8355f2b13ddf68b98461bdf292abae91e1688ee7fe7d5fc9b6a8c077ed53c638"}
Sep 08 14:35:29.103799 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:29.103765    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65fd4f6866-nr2wr" event={"ID":"fea5d1a3-561b-40e9-9978-9f1e662ef0c9","Type":"ContainerStarted","Data":"1ef1465ee481293cb53ef8192ce3f483b72c78cc04a3e6884a6606abc92d1e19"}
Sep 08 14:35:29.103936 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:29.103820    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65fd4f6866-nr2wr" event={"ID":"fea5d1a3-561b-40e9-9978-9f1e662ef0c9","Type":"ContainerStarted","Data":"0989d926900121d631b0f580e114543f0a464c790d6185d1c4d46efd05f4a512"}
Sep 08 14:35:29.104899 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:29.104882    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs" event={"ID":"3094d0b1-9ac4-42d5-824a-0fe574f90432","Type":"ContainerStarted","Data":"7180108fea10ac8fb3d3a5fcee466d7fdb38c066ad798c39ad3e862903e78188"}
Sep 08 14:35:29.119826 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:29.119787    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5879cdc854-x9vnh" podStartSLOduration=1.47538338 podStartE2EDuration="4.119774677s" totalImagesPullingTime="2.644391297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:29.119088738 +0000 UTC m=+79.846905745" watchObservedRunningTime="2026-09-08 14:35:29.119774677 +0000 UTC m=+79.847591691"
Sep 08 14:35:29.138938 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:29.138902    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-65fd4f6866-nr2wr" podStartSLOduration=33.138890718 podStartE2EDuration="33.138890718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:29.138190612 +0000 UTC m=+79.866007626" watchObservedRunningTime="2026-09-08 14:35:29.138890718 +0000 UTC m=+79.866707768"
Sep 08 14:35:29.289611 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:29.289578    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:29.292148 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:29.292122    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:30.075776 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:30.075751    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gm2hx"
Sep 08 14:35:30.110333 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:30.110285    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:30.111703 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:30.111684    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-65fd4f6866-nr2wr"
Sep 08 14:35:31.029233 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.029192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:31.114646 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.114609    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs" event={"ID":"3094d0b1-9ac4-42d5-824a-0fe574f90432","Type":"ContainerStarted","Data":"638c54d96d40e748ba2a61317da00a7ee3b7310e8fa03df6f915153622f8dc0b"}
Sep 08 14:35:31.116244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.116220    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x" event={"ID":"f371cfd8-3a5b-45d1-ad8b-04d454e09460","Type":"ContainerStarted","Data":"87d831e8ed8074d549c10f04a576fdee60f9b0dcc5bb3155b86b29581312cfbc"}
Sep 08 14:35:31.136257 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.133196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:31.141863 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.141803    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-mj4rs" podStartSLOduration=34.905850183 podStartE2EDuration="36.141789861s" totalImagesPullingTime="1.235939678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:31.140617736 +0000 UTC m=+81.868434749" watchObservedRunningTime="2026-09-08 14:35:31.141789861 +0000 UTC m=+81.869606875"
Sep 08 14:35:31.166234 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.166181    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lzg2x" podStartSLOduration=33.284111333 podStartE2EDuration="35.166170367s" totalImagesPullingTime="1.882059034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:34:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:31.165604627 +0000 UTC m=+81.893421642" watchObservedRunningTime="2026-09-08 14:35:31.166170367 +0000 UTC m=+81.893987381"
Sep 08 14:35:31.235356 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.235325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:31.249244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.249219    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf"]
Sep 08 14:35:31.251699 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.251683    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf"
Sep 08 14:35:31.254907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.254887    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 08 14:35:31.255035 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.254945    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-btfmr\""
Sep 08 14:35:31.267181 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.267158    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf"]
Sep 08 14:35:31.414563 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.414486    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:31.417571 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.417534    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ace0cfa7-8c9f-4c76-be70-0488de3d5401-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qwhhf\" (UID: \"ace0cfa7-8c9f-4c76-be70-0488de3d5401\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf"
Sep 08 14:35:31.518069 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.518044    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ace0cfa7-8c9f-4c76-be70-0488de3d5401-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qwhhf\" (UID: \"ace0cfa7-8c9f-4c76-be70-0488de3d5401\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf"
Sep 08 14:35:31.520486 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.520467    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ace0cfa7-8c9f-4c76-be70-0488de3d5401-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qwhhf\" (UID: \"ace0cfa7-8c9f-4c76-be70-0488de3d5401\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf"
Sep 08 14:35:31.562606 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.562575    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf"
Sep 08 14:35:31.611137 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.611112    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:31.684758 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.684654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:31.830654 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:31.830624    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf"]
Sep 08 14:35:31.833372 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:31.833348    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace0cfa7_8c9f_4c76_be70_0488de3d5401.slice/crio-b5d5fcd3b98ac581e6bd510eda4eebaec9b28a951fb664d9b471afabad0fc3d6 WatchSource:0}: Error finding container b5d5fcd3b98ac581e6bd510eda4eebaec9b28a951fb664d9b471afabad0fc3d6: Status 404 returned error can't find the container with id b5d5fcd3b98ac581e6bd510eda4eebaec9b28a951fb664d9b471afabad0fc3d6
Sep 08 14:35:32.121108 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:32.121070    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf" event={"ID":"ace0cfa7-8c9f-4c76-be70-0488de3d5401","Type":"ContainerStarted","Data":"b5d5fcd3b98ac581e6bd510eda4eebaec9b28a951fb664d9b471afabad0fc3d6"}
Sep 08 14:35:33.125760 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:33.125729    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf" event={"ID":"ace0cfa7-8c9f-4c76-be70-0488de3d5401","Type":"ContainerStarted","Data":"e5874d1643a4f9b30b89c4d2ef2fb3c2a3dc80c5f9d5b763a70f07d036b70bf7"}
Sep 08 14:35:33.126220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:33.125934    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf"
Sep 08 14:35:33.130750 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:33.130729    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf"
Sep 08 14:35:33.150332 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:33.150298    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qwhhf" podStartSLOduration=1.075314058 podStartE2EDuration="2.150285091s" totalImagesPullingTime="1.074971033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:33.149156139 +0000 UTC m=+83.876973153" watchObservedRunningTime="2026-09-08 14:35:33.150285091 +0000 UTC m=+83.878102104"
Sep 08 14:35:33.331240 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:33.331207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:33.333610 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:33.333590    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3c361d8-f0d5-45fb-b228-11c76fbec590-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzcsk\" (UID: \"a3c361d8-f0d5-45fb-b228-11c76fbec590\") " pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:33.346569 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:33.346543    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7h2nw\""
Sep 08 14:35:33.354078 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:33.354058    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nzcsk"
Sep 08 14:35:33.412537 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:33.412511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:33.489015 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:33.488985    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:33.628437 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:33.628368    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nzcsk"]
Sep 08 14:35:33.632814 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:33.632784    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c361d8_f0d5_45fb_b228_11c76fbec590.slice/crio-817c84de24eef01e4eec805b1782c0a0a709abc2048c616eddee2308bcabe843 WatchSource:0}: Error finding container 817c84de24eef01e4eec805b1782c0a0a709abc2048c616eddee2308bcabe843: Status 404 returned error can't find the container with id 817c84de24eef01e4eec805b1782c0a0a709abc2048c616eddee2308bcabe843
Sep 08 14:35:34.131026 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:34.130988    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzcsk" event={"ID":"a3c361d8-f0d5-45fb-b228-11c76fbec590","Type":"ContainerStarted","Data":"d9eca41e087e1cb628ef3ca599b2160f4f7e1e6f40398faf7f51b37ae0383732"}
Sep 08 14:35:34.131415 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:34.131045    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzcsk" event={"ID":"a3c361d8-f0d5-45fb-b228-11c76fbec590","Type":"ContainerStarted","Data":"817c84de24eef01e4eec805b1782c0a0a709abc2048c616eddee2308bcabe843"}
Sep 08 14:35:34.282460 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:34.282429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:35.137963 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:35.136505    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzcsk" event={"ID":"a3c361d8-f0d5-45fb-b228-11c76fbec590","Type":"ContainerStarted","Data":"56f9688cc5a541755c0dd6897cca29f6cd15846e388c4e038612ee38cb758619"}
Sep 08 14:35:36.140897 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:36.140873    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzcsk" event={"ID":"a3c361d8-f0d5-45fb-b228-11c76fbec590","Type":"ContainerStarted","Data":"2b8b6645656585b318c0cf1ff08c004c9978274d0461da18a99153f8838ed2ae"}
Sep 08 14:35:36.163965 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:36.163913    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nzcsk" podStartSLOduration=32.862773521 podStartE2EDuration="35.163899291s" totalImagesPullingTime="2.30112577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:35:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:36.162472559 +0000 UTC m=+86.890289585" watchObservedRunningTime="2026-09-08 14:35:36.163899291 +0000 UTC m=+86.891716304"
Sep 08 14:35:37.397640 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:37.397612    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:37.455380 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:37.455349    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:38.973001 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:38.972969    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:39.230518 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:39.230436    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:39.297692 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:39.297661    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:40.413569 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.413533    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:40.703898 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.703785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:40.756583 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.756549    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"]
Sep 08 14:35:40.760989 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.760970    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:40.763884 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.763862    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 08 14:35:40.764114 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.764097    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 08 14:35:40.764855 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.764820    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 14:35:40.764855 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.764850    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-t2nbh\""
Sep 08 14:35:40.774111 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.774084    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"]
Sep 08 14:35:40.787589 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.787554    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8dppw"]
Sep 08 14:35:40.791198 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.791179    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"]
Sep 08 14:35:40.791314 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.791298    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.793711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.793690    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 14:35:40.794422 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.794403    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 14:35:40.794532 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.794471    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.794701 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.794683    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 14:35:40.795026 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.795009    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v7mcr\""
Sep 08 14:35:40.799352 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.799335    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pcb58\""
Sep 08 14:35:40.800738 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.800712    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 08 14:35:40.801534 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.801520    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 08 14:35:40.801982 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.801964    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 08 14:35:40.810576 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.810558    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"]
Sep 08 14:35:40.814468 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.814445    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:40.883534 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883505    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.883534 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883538    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b7148135-b929-4dfe-b3ff-93a3205450cf-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.883755 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883556    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e407e22-2236-4c98-abd0-d4645eec70c9-sys\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.883755 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-tls\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.883755 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883623    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss2t7\" (UniqueName: \"kubernetes.io/projected/6e407e22-2236-4c98-abd0-d4645eec70c9-kube-api-access-ss2t7\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.883755 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883643    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.883755 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7148135-b929-4dfe-b3ff-93a3205450cf-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.883755 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxsk5\" (UniqueName: \"kubernetes.io/projected/fb7e4ffc-abba-4f14-b182-bd9d4987f430-kube-api-access-sxsk5\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:40.883755 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883744    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb7e4ffc-abba-4f14-b182-bd9d4987f430-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:40.884121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883767    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-textfile\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.884121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-wtmp\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.884121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883822    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb7e4ffc-abba-4f14-b182-bd9d4987f430-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:40.884121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883870    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwwr2\" (UniqueName: \"kubernetes.io/projected/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-api-access-cwwr2\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.884121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883894    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.884121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.883955    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.884121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.884001    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb7e4ffc-abba-4f14-b182-bd9d4987f430-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:40.884121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.884032    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.884416 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.884149    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e407e22-2236-4c98-abd0-d4645eec70c9-metrics-client-ca\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.884416 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.884199    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6e407e22-2236-4c98-abd0-d4645eec70c9-root\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.906739 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.906713    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:40.985194 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985166    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss2t7\" (UniqueName: \"kubernetes.io/projected/6e407e22-2236-4c98-abd0-d4645eec70c9-kube-api-access-ss2t7\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.985354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985206    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.985354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985231    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7148135-b929-4dfe-b3ff-93a3205450cf-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.985354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxsk5\" (UniqueName: \"kubernetes.io/projected/fb7e4ffc-abba-4f14-b182-bd9d4987f430-kube-api-access-sxsk5\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:40.985354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985295    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb7e4ffc-abba-4f14-b182-bd9d4987f430-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:40.985354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985319    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-textfile\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.985354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985346    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-wtmp\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985374    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb7e4ffc-abba-4f14-b182-bd9d4987f430-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985401    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwwr2\" (UniqueName: \"kubernetes.io/projected/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-api-access-cwwr2\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985461    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985491    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb7e4ffc-abba-4f14-b182-bd9d4987f430-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985520    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:40.985548    2583 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985563    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e407e22-2236-4c98-abd0-d4645eec70c9-metrics-client-ca\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985589    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6e407e22-2236-4c98-abd0-d4645eec70c9-root\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:40.985615    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb7e4ffc-abba-4f14-b182-bd9d4987f430-openshift-state-metrics-tls podName:fb7e4ffc-abba-4f14-b182-bd9d4987f430 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:41.485593128 +0000 UTC m=+92.213410136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/fb7e4ffc-abba-4f14-b182-bd9d4987f430-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-fdmj2" (UID: "fb7e4ffc-abba-4f14-b182-bd9d4987f430") : secret "openshift-state-metrics-tls" not found
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985645    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6e407e22-2236-4c98-abd0-d4645eec70c9-root\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985664    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985691    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b7148135-b929-4dfe-b3ff-93a3205450cf-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985719    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e407e22-2236-4c98-abd0-d4645eec70c9-sys\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.985781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.985745    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-tls\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.986586 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.986115    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7148135-b929-4dfe-b3ff-93a3205450cf-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.986586 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.986182    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e407e22-2236-4c98-abd0-d4645eec70c9-sys\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.986586 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.986201    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-textfile\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.986586 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.986420    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b7148135-b929-4dfe-b3ff-93a3205450cf-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.986586 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.986547    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.986866 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.986817    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb7e4ffc-abba-4f14-b182-bd9d4987f430-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:40.986866 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.986579    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-wtmp\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.986981 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.986950    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.987064 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.987029    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e407e22-2236-4c98-abd0-d4645eec70c9-metrics-client-ca\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.988344 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.988322    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb7e4ffc-abba-4f14-b182-bd9d4987f430-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:40.988678 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.988662    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-tls\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.988848 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.988812    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.989925 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.989901    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e407e22-2236-4c98-abd0-d4645eec70c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.990004 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.989991    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:40.995271 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.995250    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss2t7\" (UniqueName: \"kubernetes.io/projected/6e407e22-2236-4c98-abd0-d4645eec70c9-kube-api-access-ss2t7\") pod \"node-exporter-8dppw\" (UID: \"6e407e22-2236-4c98-abd0-d4645eec70c9\") " pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:40.996571 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:40.996554    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwwr2\" (UniqueName: \"kubernetes.io/projected/b7148135-b929-4dfe-b3ff-93a3205450cf-kube-api-access-cwwr2\") pod \"kube-state-metrics-74cb57df7-hbf82\" (UID: \"b7148135-b929-4dfe-b3ff-93a3205450cf\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:41.010414 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.010393    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxsk5\" (UniqueName: \"kubernetes.io/projected/fb7e4ffc-abba-4f14-b182-bd9d4987f430-kube-api-access-sxsk5\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:41.011384 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.011362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:41.101800 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.101771    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8dppw"
Sep 08 14:35:41.106897 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.106878    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"
Sep 08 14:35:41.117639 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:41.117592    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e407e22_2236_4c98_abd0_d4645eec70c9.slice/crio-d11eea4af8048456c10f07eda276cadd20830cd4108407825871591336e92bb5 WatchSource:0}: Error finding container d11eea4af8048456c10f07eda276cadd20830cd4108407825871591336e92bb5: Status 404 returned error can't find the container with id d11eea4af8048456c10f07eda276cadd20830cd4108407825871591336e92bb5
Sep 08 14:35:41.133178 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.133155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:41.162933 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.162306    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8dppw" event={"ID":"6e407e22-2236-4c98-abd0-d4645eec70c9","Type":"ContainerStarted","Data":"d11eea4af8048456c10f07eda276cadd20830cd4108407825871591336e92bb5"}
Sep 08 14:35:41.262014 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.261940    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:41.379035 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.379007    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:41.397881 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.397857    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hbf82"]
Sep 08 14:35:41.401172 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:41.401146    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7148135_b929_4dfe_b3ff_93a3205450cf.slice/crio-1f01ca60c245d8af9d1e865f64e3ebdc0815602db84cfc5eb0a4644862ea5bea WatchSource:0}: Error finding container 1f01ca60c245d8af9d1e865f64e3ebdc0815602db84cfc5eb0a4644862ea5bea: Status 404 returned error can't find the container with id 1f01ca60c245d8af9d1e865f64e3ebdc0815602db84cfc5eb0a4644862ea5bea
Sep 08 14:35:41.441290 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.441261    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-fgvm2"]
Sep 08 14:35:41.445684 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.445667    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-fgvm2"
Sep 08 14:35:41.451543 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.451522    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 08 14:35:41.451760 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.451742    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rhk6c\""
Sep 08 14:35:41.452205 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.452175    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 08 14:35:41.469779 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.469757    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-fgvm2"]
Sep 08 14:35:41.490407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.490383    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb7e4ffc-abba-4f14-b182-bd9d4987f430-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:41.493420 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.493397    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb7e4ffc-abba-4f14-b182-bd9d4987f430-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-fdmj2\" (UID: \"fb7e4ffc-abba-4f14-b182-bd9d4987f430\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:41.514340 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.514283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:41.590817 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.590789    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjqsz\" (UniqueName: \"kubernetes.io/projected/d918bc86-9cab-4a74-85fd-6482f08cb282-kube-api-access-fjqsz\") pod \"downloads-85d98c998f-fgvm2\" (UID: \"d918bc86-9cab-4a74-85fd-6482f08cb282\") " pod="openshift-console/downloads-85d98c998f-fgvm2"
Sep 08 14:35:41.630394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.630366    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:41.669914 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.669886    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"
Sep 08 14:35:41.692150 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.692122    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjqsz\" (UniqueName: \"kubernetes.io/projected/d918bc86-9cab-4a74-85fd-6482f08cb282-kube-api-access-fjqsz\") pod \"downloads-85d98c998f-fgvm2\" (UID: \"d918bc86-9cab-4a74-85fd-6482f08cb282\") " pod="openshift-console/downloads-85d98c998f-fgvm2"
Sep 08 14:35:41.721763 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.721735    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjqsz\" (UniqueName: \"kubernetes.io/projected/d918bc86-9cab-4a74-85fd-6482f08cb282-kube-api-access-fjqsz\") pod \"downloads-85d98c998f-fgvm2\" (UID: \"d918bc86-9cab-4a74-85fd-6482f08cb282\") " pod="openshift-console/downloads-85d98c998f-fgvm2"
Sep 08 14:35:41.755299 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.755268    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-fgvm2"
Sep 08 14:35:41.782626 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.782578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:41.953143 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.953103    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:41.980590 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:41.980507    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2"]
Sep 08 14:35:42.034574 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.034503    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:35:42.041117 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.041089    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.057510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.056536    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 08 14:35:42.057510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.056752    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9j24q\""
Sep 08 14:35:42.057510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.056777    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 08 14:35:42.057510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.056536    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 08 14:35:42.057510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.057022    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 08 14:35:42.057510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.057137    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 08 14:35:42.057510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.057318    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 08 14:35:42.057510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.057413    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 08 14:35:42.057991 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.057957    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 08 14:35:42.067332 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.067306    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 08 14:35:42.088645 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.088591    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:35:42.090308 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.090288    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-fgvm2"]
Sep 08 14:35:42.173886 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.173852    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82" event={"ID":"b7148135-b929-4dfe-b3ff-93a3205450cf","Type":"ContainerStarted","Data":"1f01ca60c245d8af9d1e865f64e3ebdc0815602db84cfc5eb0a4644862ea5bea"}
Sep 08 14:35:42.180712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.180685    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:42.197332 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197297    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-config-out\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.197471 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197338    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.197471 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197376    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.197471 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197416    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.197471 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197441    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.197676 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rknp9\" (UniqueName: \"kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-kube-api-access-rknp9\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.197676 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197503    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.197676 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197532    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.197676 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197559    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-web-config\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.197676 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197586    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.197676 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197633    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.197676 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.198012 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.197684    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.249427 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:42.249382    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7e4ffc_abba_4f14_b182_bd9d4987f430.slice/crio-993cc8539225047742133a56fc1fc52b10e21c99469b0a6a0f70b4b79080a561 WatchSource:0}: Error finding container 993cc8539225047742133a56fc1fc52b10e21c99469b0a6a0f70b4b79080a561: Status 404 returned error can't find the container with id 993cc8539225047742133a56fc1fc52b10e21c99469b0a6a0f70b4b79080a561
Sep 08 14:35:42.249770 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:42.249744    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd918bc86_9cab_4a74_85fd_6482f08cb282.slice/crio-2eb843764857a9b77e985e955a0b630ac9bc8ec635420165bf2f8bb2c631feed WatchSource:0}: Error finding container 2eb843764857a9b77e985e955a0b630ac9bc8ec635420165bf2f8bb2c631feed: Status 404 returned error can't find the container with id 2eb843764857a9b77e985e955a0b630ac9bc8ec635420165bf2f8bb2c631feed
Sep 08 14:35:42.298826 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.298732    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rknp9\" (UniqueName: \"kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-kube-api-access-rknp9\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.298826 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.298780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.298826 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.298813    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.299059 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.298869    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-web-config\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.299059 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.298895    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.299059 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.298937    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.299059 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.298965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.299059 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.298989    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.299059 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.299041    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-config-out\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.299342 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.299068    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.299342 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.299118    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.299342 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.299161    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.299342 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.299184    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.299342 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:42.299310    2583 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 08 14:35:42.299575 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:35:42.299366    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-main-tls podName:241686e6-f732-43d5-b6fc-d9eb922215c9 nodeName:}" failed. No retries permitted until 2026-09-08 14:35:42.799347279 +0000 UTC m=+93.527164286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9") : secret "alertmanager-main-tls" not found
Sep 08 14:35:42.299740 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.299657    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.301798 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.300576    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.301798 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.301421    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.302498 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.302473    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.303039 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.303019    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.303484 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.303462    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-web-config\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.304033 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.304003    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.304487 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.304461    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.305348 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.305310    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.306533 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.306510    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-config-out\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.307569 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.307541    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.346399 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.346317    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rknp9\" (UniqueName: \"kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-kube-api-access-rknp9\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.426912 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.426880    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:42.522453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.522423    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:42.804093 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.803992    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.807039 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.807017    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:42.953974 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:42.953930    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:35:43.170048 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.170014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:43.186409 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.186317    2583 generic.go:356] "Generic (PLEG): container finished" podID="6e407e22-2236-4c98-abd0-d4645eec70c9" containerID="ad9e387b24e852587b6c835dc2fed5e6a610be3430c3ee063b5bb088187757ca" exitCode=0
Sep 08 14:35:43.186409 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.186376    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8dppw" event={"ID":"6e407e22-2236-4c98-abd0-d4645eec70c9","Type":"ContainerDied","Data":"ad9e387b24e852587b6c835dc2fed5e6a610be3430c3ee063b5bb088187757ca"}
Sep 08 14:35:43.189738 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.189707    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2" event={"ID":"fb7e4ffc-abba-4f14-b182-bd9d4987f430","Type":"ContainerStarted","Data":"bbb51499a2fec4ce4be92aeb4fcb5d43cea21384cc7a23a7084a14e14c2473c8"}
Sep 08 14:35:43.189882 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.189773    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2" event={"ID":"fb7e4ffc-abba-4f14-b182-bd9d4987f430","Type":"ContainerStarted","Data":"164448be1cf43db200cbd6e2646ed60eadd5a0b50f7087f14d14252f6a5f70d6"}
Sep 08 14:35:43.189882 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.189794    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2" event={"ID":"fb7e4ffc-abba-4f14-b182-bd9d4987f430","Type":"ContainerStarted","Data":"993cc8539225047742133a56fc1fc52b10e21c99469b0a6a0f70b4b79080a561"}
Sep 08 14:35:43.193227 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.193200    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82" event={"ID":"b7148135-b929-4dfe-b3ff-93a3205450cf","Type":"ContainerStarted","Data":"ac01151c6f076e099d2a9ab55785eb0481160572afc097a5cb863f7208a33c1b"}
Sep 08 14:35:43.193329 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.193255    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82" event={"ID":"b7148135-b929-4dfe-b3ff-93a3205450cf","Type":"ContainerStarted","Data":"19dbead09d0f8cc243964ccb761f1c9915b8c37df991041cde83f0249ed1463d"}
Sep 08 14:35:43.193329 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.193287    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82" event={"ID":"b7148135-b929-4dfe-b3ff-93a3205450cf","Type":"ContainerStarted","Data":"66bd136c01f030ed5682c0fba5b9b90055ef700d0eacdbb924dd553eb754dfc5"}
Sep 08 14:35:43.194734 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.194704    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-fgvm2" event={"ID":"d918bc86-9cab-4a74-85fd-6482f08cb282","Type":"ContainerStarted","Data":"2eb843764857a9b77e985e955a0b630ac9bc8ec635420165bf2f8bb2c631feed"}
Sep 08 14:35:43.237148 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.237092    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hbf82" podStartSLOduration=1.9206393419999999 podStartE2EDuration="3.237073036s" totalImagesPullingTime="1.316433694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:43.235048673 +0000 UTC m=+93.962865686" watchObservedRunningTime="2026-09-08 14:35:43.237073036 +0000 UTC m=+93.964890050"
Sep 08 14:35:43.262056 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.261940    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:35:43.267855 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.267789    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:43.343214 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.343182    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:43.817055 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:43.817017    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:44.022088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:44.022057    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:44.200166 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:44.200086    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerStarted","Data":"9d599e6c86f5dd0668d0f5e12a201f25c033bdd9abd0a182510a2a61e9fc1e81"}
Sep 08 14:35:44.202354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:44.202327    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2" event={"ID":"fb7e4ffc-abba-4f14-b182-bd9d4987f430","Type":"ContainerStarted","Data":"7f4d0782ea9cfdcf40344d2f587d9774e883fc542676ba29e1c11b9c6b037e98"}
Sep 08 14:35:44.204600 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:44.204576    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8dppw" event={"ID":"6e407e22-2236-4c98-abd0-d4645eec70c9","Type":"ContainerStarted","Data":"1366d2aacc8486c6ab1adc680ed6b5433ae4c5360052ab5dc83277ac79a4faba"}
Sep 08 14:35:44.204680 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:44.204621    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8dppw" event={"ID":"6e407e22-2236-4c98-abd0-d4645eec70c9","Type":"ContainerStarted","Data":"253ac0202c8be66e7110ee64bea0945fb4add6dcf186c510bcc6e69810a3e04a"}
Sep 08 14:35:44.222869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:44.222813    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:44.229914 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:44.229872    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fdmj2" podStartSLOduration=3.051223785 podStartE2EDuration="4.229857279s" totalImagesPullingTime="1.178633494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:44.228153479 +0000 UTC m=+94.955970494" watchObservedRunningTime="2026-09-08 14:35:44.229857279 +0000 UTC m=+94.957674287"
Sep 08 14:35:44.356455 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:44.356417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:44.455453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:44.455370    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:45.435366 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:45.435265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:45.523277 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:45.523241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:45.796490 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:45.796458    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:46.215013 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:46.214927    2583 generic.go:356] "Generic (PLEG): container finished" podID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerID="d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463" exitCode=0
Sep 08 14:35:46.215159 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:46.215023    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerDied","Data":"d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463"}
Sep 08 14:35:46.246950 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:46.246897    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8dppw" podStartSLOduration=4.988794257 podStartE2EDuration="6.246881035s" totalImagesPullingTime="1.180276869s" totalInitContainerRuntime="77.809909ms" isStatefulPod=true podCreationTimestamp="2026-09-08 14:35:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:44.303098446 +0000 UTC m=+95.030915460" watchObservedRunningTime="2026-09-08 14:35:46.246881035 +0000 UTC m=+96.974698045"
Sep 08 14:35:46.558524 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:46.558491    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:46.665127 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:46.665095    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:47.052463 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:47.052431    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-php6q"
Sep 08 14:35:47.300584 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:47.300548    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:47.428047 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:47.427961    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:48.319443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:48.319415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:49.236592 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:49.236556    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerStarted","Data":"792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748"}
Sep 08 14:35:49.236750 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:49.236645    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerStarted","Data":"581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281"}
Sep 08 14:35:49.236750 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:49.236677    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerStarted","Data":"e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4"}
Sep 08 14:35:49.236750 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:49.236700    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerStarted","Data":"16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1"}
Sep 08 14:35:49.236750 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:49.236722    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerStarted","Data":"06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873"}
Sep 08 14:35:50.091552 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:50.091523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:50.191455 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:50.191416    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:50.246248 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:50.246215    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerStarted","Data":"8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb"}
Sep 08 14:35:50.284845 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:50.284778    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.116266828 podStartE2EDuration="8.284756108s" totalImagesPullingTime="5.152093354s" totalInitContainerRuntime="16.395926ms" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:50.277933424 +0000 UTC m=+101.005750439" watchObservedRunningTime="2026-09-08 14:35:50.284756108 +0000 UTC m=+101.012573121"
Sep 08 14:35:51.050134 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.050097    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:51.155322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.155288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:51.263024 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.262994    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:51.354786 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.354701    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:51.786224 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.786191    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7679995c79-ng9pw"]
Sep 08 14:35:51.800385 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.800358    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.803623 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.803600    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 08 14:35:51.803789 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.803633    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 08 14:35:51.805078 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.805045    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 08 14:35:51.805294 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.805278    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 08 14:35:51.806105 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.805593    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 08 14:35:51.806105 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.805805    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pcvkg\""
Sep 08 14:35:51.807188 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.807162    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7679995c79-ng9pw"]
Sep 08 14:35:51.881851 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.881793    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-serving-cert\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.882011 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.881864    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-console-config\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.882011 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.881904    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-oauth-serving-cert\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.882011 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.881952    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-service-ca\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.882011 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.882000    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk9mp\" (UniqueName: \"kubernetes.io/projected/93cc1951-74b4-4903-86a3-4b615644d735-kube-api-access-bk9mp\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.882174 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.882030    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-oauth-config\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.983235 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.983192    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-console-config\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.983512 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.983248    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-oauth-serving-cert\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.983512 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.983286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-service-ca\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.983512 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.983324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk9mp\" (UniqueName: \"kubernetes.io/projected/93cc1951-74b4-4903-86a3-4b615644d735-kube-api-access-bk9mp\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.983512 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.983367    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-oauth-config\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.983512 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.983400    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-serving-cert\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.984065 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.984035    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-console-config\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.984188 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.984064    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-oauth-serving-cert\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.984188 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.984079    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-service-ca\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.986332 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.986307    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-serving-cert\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:51.986597 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:51.986574    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-oauth-config\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:52.001682 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:52.001655    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk9mp\" (UniqueName: \"kubernetes.io/projected/93cc1951-74b4-4903-86a3-4b615644d735-kube-api-access-bk9mp\") pod \"console-7679995c79-ng9pw\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") " pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:52.113269 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:52.113187    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:35:55.565954 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:55.565918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:55.631386 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:55.631346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:58.950514 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:58.950474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:59.009621 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.009586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:35:59.255762 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.255728    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7679995c79-ng9pw"]
Sep 08 14:35:59.260569 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:35:59.260540    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93cc1951_74b4_4903_86a3_4b615644d735.slice/crio-16790a91bdefee0ceeab438f4371f72372e4b5419ec5e3711f847a7645878c87 WatchSource:0}: Error finding container 16790a91bdefee0ceeab438f4371f72372e4b5419ec5e3711f847a7645878c87: Status 404 returned error can't find the container with id 16790a91bdefee0ceeab438f4371f72372e4b5419ec5e3711f847a7645878c87
Sep 08 14:35:59.287309 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.287217    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7679995c79-ng9pw" event={"ID":"93cc1951-74b4-4903-86a3-4b615644d735","Type":"ContainerStarted","Data":"16790a91bdefee0ceeab438f4371f72372e4b5419ec5e3711f847a7645878c87"}
Sep 08 14:35:59.288954 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.288905    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-fgvm2" event={"ID":"d918bc86-9cab-4a74-85fd-6482f08cb282","Type":"ContainerStarted","Data":"78980ccbaab88c54749a5fdf47e889c65d05654ada9ef0f56d0f491c4edf064f"}
Sep 08 14:35:59.289665 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.289593    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-fgvm2"
Sep 08 14:35:59.297438 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.297420    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-fgvm2"
Sep 08 14:35:59.313588 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.313538    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-fgvm2" podStartSLOduration=1.5039339699999998 podStartE2EDuration="18.31352036s" totalImagesPullingTime="16.80958639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:35:59.31218659 +0000 UTC m=+110.040003605" watchObservedRunningTime="2026-09-08 14:35:59.31352036 +0000 UTC m=+110.041337375"
Sep 08 14:35:59.743364 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.743327    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59d5974b85-kdfhj"]
Sep 08 14:35:59.746777 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.746752    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.757995 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.757969    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 08 14:35:59.769873 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.769846    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59d5974b85-kdfhj"]
Sep 08 14:35:59.852217 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.852174    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-serving-cert\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.852217 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.852220    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-oauth-serving-cert\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.852460 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.852245    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gklmq\" (UniqueName: \"kubernetes.io/projected/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-kube-api-access-gklmq\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.852460 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.852356    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-service-ca\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.852460 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.852404    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-config\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.852460 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.852443    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-trusted-ca-bundle\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.852658 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.852497    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-oauth-config\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.953082 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.953040    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-trusted-ca-bundle\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.953539 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.953111    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-oauth-config\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.953539 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.953177    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-serving-cert\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.953539 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.953208    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-oauth-serving-cert\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.953539 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.953240    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gklmq\" (UniqueName: \"kubernetes.io/projected/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-kube-api-access-gklmq\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.953539 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.953273    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-service-ca\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.953539 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.953302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-config\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.954023 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.953996    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-config\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.954117 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.954091    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-trusted-ca-bundle\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.954768 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.954743    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-oauth-serving-cert\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.955679 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.955650    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-service-ca\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.957020 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.956997    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-oauth-config\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.957626 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.957607    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-serving-cert\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:35:59.964679 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:35:59.964653    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gklmq\" (UniqueName: \"kubernetes.io/projected/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-kube-api-access-gklmq\") pod \"console-59d5974b85-kdfhj\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") " pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:36:00.059178 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:00.059081    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:36:00.238976 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:00.238882    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:00.298311 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:00.298274    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:00.371036 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:00.370961    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59d5974b85-kdfhj"]
Sep 08 14:36:00.382879 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:36:00.382807    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c769e47_42d4_4d29_bdc0_0c062d1df5ea.slice/crio-753c7570e6db785375b1c1c816a3b04ce42ef060c80964dd6ba42c9a41ce936c WatchSource:0}: Error finding container 753c7570e6db785375b1c1c816a3b04ce42ef060c80964dd6ba42c9a41ce936c: Status 404 returned error can't find the container with id 753c7570e6db785375b1c1c816a3b04ce42ef060c80964dd6ba42c9a41ce936c
Sep 08 14:36:01.304855 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:01.304811    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d5974b85-kdfhj" event={"ID":"6c769e47-42d4-4d29-bdc0-0c062d1df5ea","Type":"ContainerStarted","Data":"753c7570e6db785375b1c1c816a3b04ce42ef060c80964dd6ba42c9a41ce936c"}
Sep 08 14:36:02.076781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:02.076741    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:02.176137 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:02.176068    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:03.905323 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:03.905280    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:04.013000 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:04.012769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:04.322084 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:04.322045    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7679995c79-ng9pw" event={"ID":"93cc1951-74b4-4903-86a3-4b615644d735","Type":"ContainerStarted","Data":"7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4"}
Sep 08 14:36:04.324258 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:04.324227    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d5974b85-kdfhj" event={"ID":"6c769e47-42d4-4d29-bdc0-0c062d1df5ea","Type":"ContainerStarted","Data":"b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a"}
Sep 08 14:36:04.358823 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:04.358768    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7679995c79-ng9pw" podStartSLOduration=8.696198272 podStartE2EDuration="13.358752475s" totalImagesPullingTime="4.662554203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:36:04.358508458 +0000 UTC m=+115.086325473" watchObservedRunningTime="2026-09-08 14:36:04.358752475 +0000 UTC m=+115.086569489"
Sep 08 14:36:04.389256 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:04.389212    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59d5974b85-kdfhj" podStartSLOduration=1.840945204 podStartE2EDuration="5.389193041s" totalImagesPullingTime="3.548247837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:36:04.388443498 +0000 UTC m=+115.116260512" watchObservedRunningTime="2026-09-08 14:36:04.389193041 +0000 UTC m=+115.117010056"
Sep 08 14:36:06.029420 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:06.029382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:06.128680 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:06.128639    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:08.296457 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:08.296413    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:08.397925 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:08.397883    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:10.060126 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:10.060094    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:36:10.060558 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:10.060157    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:36:10.065065 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:10.065045    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:36:10.354456 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:10.354382    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:36:10.436509 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:10.436476    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7679995c79-ng9pw"]
Sep 08 14:36:10.594517 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:10.594482    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:10.696305 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:10.696223    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:10.796282 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:10.796255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:11.165876 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:11.165849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:11.237822 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:11.237788    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:11.548754 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:11.548726    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:11.648785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:11.648750    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:12.113464 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:12.113433    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:36:21.017638 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:21.017604    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:21.088426 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:21.088395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:35.457511 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.457467    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7679995c79-ng9pw" podUID="93cc1951-74b4-4903-86a3-4b615644d735" containerName="console" containerID="cri-o://7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4" gracePeriod=15
Sep 08 14:36:35.738454 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.738427    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:36:35.865914 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.865885    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk9mp\" (UniqueName: \"kubernetes.io/projected/93cc1951-74b4-4903-86a3-4b615644d735-kube-api-access-bk9mp\") pod \"93cc1951-74b4-4903-86a3-4b615644d735\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") "
Sep 08 14:36:35.866067 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.865927    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-console-config\") pod \"93cc1951-74b4-4903-86a3-4b615644d735\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") "
Sep 08 14:36:35.866067 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.865949    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-serving-cert\") pod \"93cc1951-74b4-4903-86a3-4b615644d735\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") "
Sep 08 14:36:35.866067 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.865980    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-oauth-config\") pod \"93cc1951-74b4-4903-86a3-4b615644d735\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") "
Sep 08 14:36:35.866067 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.866017    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-service-ca\") pod \"93cc1951-74b4-4903-86a3-4b615644d735\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") "
Sep 08 14:36:35.866278 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.866081    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-oauth-serving-cert\") pod \"93cc1951-74b4-4903-86a3-4b615644d735\" (UID: \"93cc1951-74b4-4903-86a3-4b615644d735\") "
Sep 08 14:36:35.866455 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.866419    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-console-config" (OuterVolumeSpecName: "console-config") pod "93cc1951-74b4-4903-86a3-4b615644d735" (UID: "93cc1951-74b4-4903-86a3-4b615644d735"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:36:35.866541 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.866483    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-service-ca" (OuterVolumeSpecName: "service-ca") pod "93cc1951-74b4-4903-86a3-4b615644d735" (UID: "93cc1951-74b4-4903-86a3-4b615644d735"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:36:35.866598 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.866538    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "93cc1951-74b4-4903-86a3-4b615644d735" (UID: "93cc1951-74b4-4903-86a3-4b615644d735"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:36:35.868328 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.868294    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "93cc1951-74b4-4903-86a3-4b615644d735" (UID: "93cc1951-74b4-4903-86a3-4b615644d735"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:36:35.868422 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.868327    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "93cc1951-74b4-4903-86a3-4b615644d735" (UID: "93cc1951-74b4-4903-86a3-4b615644d735"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:36:35.868422 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.868299    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93cc1951-74b4-4903-86a3-4b615644d735-kube-api-access-bk9mp" (OuterVolumeSpecName: "kube-api-access-bk9mp") pod "93cc1951-74b4-4903-86a3-4b615644d735" (UID: "93cc1951-74b4-4903-86a3-4b615644d735"). InnerVolumeSpecName "kube-api-access-bk9mp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:36:35.967002 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.966973    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk9mp\" (UniqueName: \"kubernetes.io/projected/93cc1951-74b4-4903-86a3-4b615644d735-kube-api-access-bk9mp\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:36:35.967002 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.966998    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-console-config\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:36:35.967002 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.967009    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-serving-cert\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:36:35.967209 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.967018    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93cc1951-74b4-4903-86a3-4b615644d735-console-oauth-config\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:36:35.967209 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.967026    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:36:35.967209 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:35.967034    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93cc1951-74b4-4903-86a3-4b615644d735-oauth-serving-cert\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:36:36.453878 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:36.453822    2583 generic.go:356] "Generic (PLEG): container finished" podID="93cc1951-74b4-4903-86a3-4b615644d735" containerID="7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4" exitCode=0
Sep 08 14:36:36.454032 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:36.453889    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7679995c79-ng9pw"
Sep 08 14:36:36.454032 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:36.453906    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7679995c79-ng9pw" event={"ID":"93cc1951-74b4-4903-86a3-4b615644d735","Type":"ContainerDied","Data":"7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4"}
Sep 08 14:36:36.454032 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:36.453957    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7679995c79-ng9pw" event={"ID":"93cc1951-74b4-4903-86a3-4b615644d735","Type":"ContainerDied","Data":"16790a91bdefee0ceeab438f4371f72372e4b5419ec5e3711f847a7645878c87"}
Sep 08 14:36:36.454032 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:36.453977    2583 scope.go:117] "RemoveContainer" containerID="7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4"
Sep 08 14:36:36.462714 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:36.462587    2583 scope.go:117] "RemoveContainer" containerID="7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4"
Sep 08 14:36:36.462962 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:36:36.462874    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4\": container with ID starting with 7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4 not found: ID does not exist" containerID="7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4"
Sep 08 14:36:36.462962 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:36.462898    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4"} err="failed to get container status \"7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4\": rpc error: code = NotFound desc = could not find container \"7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4\": container with ID starting with 7604f85e0327654032d4662f53d36e9c2a2dd6c53f2eb03eecfa4fe1f0e9e9c4 not found: ID does not exist"
Sep 08 14:36:36.475171 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:36.475151    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7679995c79-ng9pw"]
Sep 08 14:36:36.479823 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:36.479804    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7679995c79-ng9pw"]
Sep 08 14:36:37.791678 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:37.791648    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93cc1951-74b4-4903-86a3-4b615644d735" path="/var/lib/kubelet/pods/93cc1951-74b4-4903-86a3-4b615644d735/volumes"
Sep 08 14:36:56.989302 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:56.989263    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:36:57.047165 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:36:57.047131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:01.224998 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.224917    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:37:01.225355 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.225280    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="alertmanager" containerID="cri-o://06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873" gracePeriod=120
Sep 08 14:37:01.225411 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.225349    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy-metric" containerID="cri-o://792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748" gracePeriod=120
Sep 08 14:37:01.225411 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.225374    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="prom-label-proxy" containerID="cri-o://8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb" gracePeriod=120
Sep 08 14:37:01.225519 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.225413    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy-web" containerID="cri-o://e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4" gracePeriod=120
Sep 08 14:37:01.225519 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.225443    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="config-reloader" containerID="cri-o://16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1" gracePeriod=120
Sep 08 14:37:01.225519 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.225414    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy" containerID="cri-o://581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281" gracePeriod=120
Sep 08 14:37:01.567145 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.567060    2583 generic.go:356] "Generic (PLEG): container finished" podID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerID="8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb" exitCode=0
Sep 08 14:37:01.567145 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.567091    2583 generic.go:356] "Generic (PLEG): container finished" podID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerID="581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281" exitCode=0
Sep 08 14:37:01.567145 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.567099    2583 generic.go:356] "Generic (PLEG): container finished" podID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerID="16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1" exitCode=0
Sep 08 14:37:01.567145 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.567097    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerDied","Data":"8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb"}
Sep 08 14:37:01.567400 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.567158    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerDied","Data":"581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281"}
Sep 08 14:37:01.567400 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.567108    2583 generic.go:356] "Generic (PLEG): container finished" podID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerID="06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873" exitCode=0
Sep 08 14:37:01.567400 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.567190    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerDied","Data":"16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1"}
Sep 08 14:37:01.567400 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:01.567228    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerDied","Data":"06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873"}
Sep 08 14:37:02.469666 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.469643    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.553563 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553491    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-main-tls\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.553563 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553525    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-config-volume\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.553563 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553546    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-config-out\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.553563 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553562    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-trusted-ca-bundle\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.553910 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553578    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-cluster-tls-config\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.553910 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553650    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.553910 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553683    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.553910 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553717    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rknp9\" (UniqueName: \"kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-kube-api-access-rknp9\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.553910 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553761    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-web-config\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.553910 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553787    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-tls-assets\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.553910 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553813    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.553910 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553863    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-main-db\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.554316 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553918    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-metrics-client-ca\") pod \"241686e6-f732-43d5-b6fc-d9eb922215c9\" (UID: \"241686e6-f732-43d5-b6fc-d9eb922215c9\") "
Sep 08 14:37:02.554316 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.553985    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:37:02.555376 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.554453    2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.555376 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.554727    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:37:02.555376 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.555325    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:37:02.557785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.557749    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-config-volume" (OuterVolumeSpecName: "config-volume") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:37:02.558289 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.558164    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-config-out" (OuterVolumeSpecName: "config-out") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:37:02.558289 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.558255    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:37:02.558538 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.558488    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:37:02.559237 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.558792    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:37:02.559237 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.559102    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:37:02.559712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.559667    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-kube-api-access-rknp9" (OuterVolumeSpecName: "kube-api-access-rknp9") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "kube-api-access-rknp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:37:02.560194 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.560162    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:37:02.563023 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.562765    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:37:02.569583 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.569564    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-web-config" (OuterVolumeSpecName: "web-config") pod "241686e6-f732-43d5-b6fc-d9eb922215c9" (UID: "241686e6-f732-43d5-b6fc-d9eb922215c9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:37:02.574125 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.574103    2583 generic.go:356] "Generic (PLEG): container finished" podID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerID="792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748" exitCode=0
Sep 08 14:37:02.574125 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.574124    2583 generic.go:356] "Generic (PLEG): container finished" podID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerID="e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4" exitCode=0
Sep 08 14:37:02.574264 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.574170    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.574264 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.574178    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerDied","Data":"792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748"}
Sep 08 14:37:02.574264 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.574250    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerDied","Data":"e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4"}
Sep 08 14:37:02.574372 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.574281    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241686e6-f732-43d5-b6fc-d9eb922215c9","Type":"ContainerDied","Data":"9d599e6c86f5dd0668d0f5e12a201f25c033bdd9abd0a182510a2a61e9fc1e81"}
Sep 08 14:37:02.574372 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.574312    2583 scope.go:117] "RemoveContainer" containerID="8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb"
Sep 08 14:37:02.582621 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.582604    2583 scope.go:117] "RemoveContainer" containerID="792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748"
Sep 08 14:37:02.589562 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.589547    2583 scope.go:117] "RemoveContainer" containerID="581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281"
Sep 08 14:37:02.599171 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.599147    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:37:02.600101 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.600085    2583 scope.go:117] "RemoveContainer" containerID="e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4"
Sep 08 14:37:02.605264 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.605121    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:37:02.611882 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.611863    2583 scope.go:117] "RemoveContainer" containerID="16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1"
Sep 08 14:37:02.619819 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.619800    2583 scope.go:117] "RemoveContainer" containerID="06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873"
Sep 08 14:37:02.627228 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.627211    2583 scope.go:117] "RemoveContainer" containerID="d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463"
Sep 08 14:37:02.634212 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.634193    2583 scope.go:117] "RemoveContainer" containerID="8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb"
Sep 08 14:37:02.634492 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:37:02.634471    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb\": container with ID starting with 8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb not found: ID does not exist" containerID="8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb"
Sep 08 14:37:02.634539 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.634504    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb"} err="failed to get container status \"8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb\": rpc error: code = NotFound desc = could not find container \"8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb\": container with ID starting with 8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb not found: ID does not exist"
Sep 08 14:37:02.634539 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.634522    2583 scope.go:117] "RemoveContainer" containerID="792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748"
Sep 08 14:37:02.634787 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:37:02.634769    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748\": container with ID starting with 792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748 not found: ID does not exist" containerID="792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748"
Sep 08 14:37:02.634850 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.634792    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748"} err="failed to get container status \"792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748\": rpc error: code = NotFound desc = could not find container \"792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748\": container with ID starting with 792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748 not found: ID does not exist"
Sep 08 14:37:02.634850 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.634807    2583 scope.go:117] "RemoveContainer" containerID="581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281"
Sep 08 14:37:02.635044 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:37:02.635028    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281\": container with ID starting with 581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281 not found: ID does not exist" containerID="581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281"
Sep 08 14:37:02.635105 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.635052    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281"} err="failed to get container status \"581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281\": rpc error: code = NotFound desc = could not find container \"581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281\": container with ID starting with 581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281 not found: ID does not exist"
Sep 08 14:37:02.635105 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.635073    2583 scope.go:117] "RemoveContainer" containerID="e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4"
Sep 08 14:37:02.635318 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:37:02.635304    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4\": container with ID starting with e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4 not found: ID does not exist" containerID="e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4"
Sep 08 14:37:02.635355 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.635321    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4"} err="failed to get container status \"e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4\": rpc error: code = NotFound desc = could not find container \"e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4\": container with ID starting with e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4 not found: ID does not exist"
Sep 08 14:37:02.635355 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.635337    2583 scope.go:117] "RemoveContainer" containerID="16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1"
Sep 08 14:37:02.635542 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:37:02.635526    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1\": container with ID starting with 16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1 not found: ID does not exist" containerID="16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1"
Sep 08 14:37:02.635579 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.635548    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1"} err="failed to get container status \"16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1\": rpc error: code = NotFound desc = could not find container \"16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1\": container with ID starting with 16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1 not found: ID does not exist"
Sep 08 14:37:02.635579 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.635561    2583 scope.go:117] "RemoveContainer" containerID="06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873"
Sep 08 14:37:02.635805 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:37:02.635791    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873\": container with ID starting with 06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873 not found: ID does not exist" containerID="06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873"
Sep 08 14:37:02.635877 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.635807    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873"} err="failed to get container status \"06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873\": rpc error: code = NotFound desc = could not find container \"06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873\": container with ID starting with 06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873 not found: ID does not exist"
Sep 08 14:37:02.635877 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.635818    2583 scope.go:117] "RemoveContainer" containerID="d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463"
Sep 08 14:37:02.636116 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:37:02.636100    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463\": container with ID starting with d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463 not found: ID does not exist" containerID="d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463"
Sep 08 14:37:02.636161 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.636120    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463"} err="failed to get container status \"d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463\": rpc error: code = NotFound desc = could not find container \"d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463\": container with ID starting with d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463 not found: ID does not exist"
Sep 08 14:37:02.636161 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.636133    2583 scope.go:117] "RemoveContainer" containerID="8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb"
Sep 08 14:37:02.636336 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.636318    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb"} err="failed to get container status \"8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb\": rpc error: code = NotFound desc = could not find container \"8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb\": container with ID starting with 8f6a51bca8178238683f5477d1b4aec8d873f31ec6bdb260aba8f078291052bb not found: ID does not exist"
Sep 08 14:37:02.636379 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.636336    2583 scope.go:117] "RemoveContainer" containerID="792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748"
Sep 08 14:37:02.636518 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.636503    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748"} err="failed to get container status \"792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748\": rpc error: code = NotFound desc = could not find container \"792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748\": container with ID starting with 792b5fd23d1ee5105643e1b443e8cd1114b9a27d002eb8955f1dc2698b8ae748 not found: ID does not exist"
Sep 08 14:37:02.636558 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.636518    2583 scope.go:117] "RemoveContainer" containerID="581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281"
Sep 08 14:37:02.636711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.636696    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281"} err="failed to get container status \"581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281\": rpc error: code = NotFound desc = could not find container \"581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281\": container with ID starting with 581c13377a6c141599a221e17da7cf1f997f413908e8bc0a7c7909f14ddf4281 not found: ID does not exist"
Sep 08 14:37:02.636759 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.636711    2583 scope.go:117] "RemoveContainer" containerID="e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4"
Sep 08 14:37:02.636920 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.636904    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4"} err="failed to get container status \"e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4\": rpc error: code = NotFound desc = could not find container \"e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4\": container with ID starting with e330ffbf3be2ea5623bf6232f997b6d615dfa333fb20e779b6992b57a59e2be4 not found: ID does not exist"
Sep 08 14:37:02.636966 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.636919    2583 scope.go:117] "RemoveContainer" containerID="16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1"
Sep 08 14:37:02.637089 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.637073    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1"} err="failed to get container status \"16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1\": rpc error: code = NotFound desc = could not find container \"16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1\": container with ID starting with 16718cb051ba908639fc16eee68996552c5914c1671238fa5a5dd901ff04def1 not found: ID does not exist"
Sep 08 14:37:02.637132 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.637089    2583 scope.go:117] "RemoveContainer" containerID="06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873"
Sep 08 14:37:02.637276 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.637261    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873"} err="failed to get container status \"06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873\": rpc error: code = NotFound desc = could not find container \"06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873\": container with ID starting with 06b8d88f0cbfd3ceb6ba36a898d90161b6c1c391f61b2af4d598c28c3df93873 not found: ID does not exist"
Sep 08 14:37:02.637315 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.637276    2583 scope.go:117] "RemoveContainer" containerID="d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463"
Sep 08 14:37:02.637487 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.637470    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463"} err="failed to get container status \"d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463\": rpc error: code = NotFound desc = could not find container \"d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463\": container with ID starting with d1d676d4fe1b1dd29adeb56d789ed3604f3863fbc2b641e9eee956a0414f1463 not found: ID does not exist"
Sep 08 14:37:02.643596 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643576    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:37:02.643894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643871    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="prom-label-proxy"
Sep 08 14:37:02.643894 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643896    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="prom-label-proxy"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643907    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="init-config-reloader"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643912    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="init-config-reloader"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643922    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="config-reloader"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643928    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="config-reloader"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643933    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643939    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643945    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="alertmanager"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643950    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="alertmanager"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643955    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy-web"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643960    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy-web"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643966    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy-metric"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643970    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy-metric"
Sep 08 14:37:02.643977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643981    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93cc1951-74b4-4903-86a3-4b615644d735" containerName="console"
Sep 08 14:37:02.644334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.643986    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="93cc1951-74b4-4903-86a3-4b615644d735" containerName="console"
Sep 08 14:37:02.644334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.644030    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="config-reloader"
Sep 08 14:37:02.644334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.644036    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy"
Sep 08 14:37:02.644334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.644042    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="alertmanager"
Sep 08 14:37:02.644334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.644047    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="prom-label-proxy"
Sep 08 14:37:02.644334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.644052    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy-web"
Sep 08 14:37:02.644334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.644058    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" containerName="kube-rbac-proxy-metric"
Sep 08 14:37:02.644334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.644064    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="93cc1951-74b4-4903-86a3-4b615644d735" containerName="console"
Sep 08 14:37:02.647940 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.647925    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.652967 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.652946    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 08 14:37:02.653344 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.653329    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 08 14:37:02.655706 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655683    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-main-tls\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.655805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655711    2583 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.655805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655724    2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-config-out\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.655805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655737    2583 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-cluster-tls-config\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.655805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655749    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.655805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655758    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.655805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655767    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rknp9\" (UniqueName: \"kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-kube-api-access-rknp9\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.655805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655778    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 08 14:37:02.655805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655795    2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-web-config\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.655805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655810    2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241686e6-f732-43d5-b6fc-d9eb922215c9-tls-assets\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.656220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655822    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241686e6-f732-43d5-b6fc-d9eb922215c9-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.656220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655850    2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/241686e6-f732-43d5-b6fc-d9eb922215c9-alertmanager-main-db\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.656220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655863    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241686e6-f732-43d5-b6fc-d9eb922215c9-metrics-client-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:37:02.656220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655876    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 08 14:37:02.656220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655908    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 08 14:37:02.656220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655956    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 08 14:37:02.656220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.655991    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 08 14:37:02.656220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.656034    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 08 14:37:02.656441 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.656268    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9j24q\""
Sep 08 14:37:02.660063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.660046    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 08 14:37:02.678495 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.678472    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:37:02.756497 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756470    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2f29701-c5b7-465a-9b76-8143abd61e92-config-out\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756590 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756517    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756590 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756665 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756586    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2f29701-c5b7-465a-9b76-8143abd61e92-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756699 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756660    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvzgf\" (UniqueName: \"kubernetes.io/projected/d2f29701-c5b7-465a-9b76-8143abd61e92-kube-api-access-mvzgf\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756699 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756812 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756717    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-web-config\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756812 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756732    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2f29701-c5b7-465a-9b76-8143abd61e92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756812 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756969 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756851    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756969 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756875    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d2f29701-c5b7-465a-9b76-8143abd61e92-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756969 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756890    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2f29701-c5b7-465a-9b76-8143abd61e92-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.756969 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.756909    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-config-volume\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858118 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858046    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2f29701-c5b7-465a-9b76-8143abd61e92-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858118 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858092    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvzgf\" (UniqueName: \"kubernetes.io/projected/d2f29701-c5b7-465a-9b76-8143abd61e92-kube-api-access-mvzgf\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858118 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858111    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858346 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858166    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-web-config\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858346 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858193    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2f29701-c5b7-465a-9b76-8143abd61e92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858346 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858226    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858346 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858252    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858346 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858276    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d2f29701-c5b7-465a-9b76-8143abd61e92-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858346 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858299    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2f29701-c5b7-465a-9b76-8143abd61e92-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858346 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-config-volume\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858346 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858349    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2f29701-c5b7-465a-9b76-8143abd61e92-config-out\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858730 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858385    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858730 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858418    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.858730 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.858679    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d2f29701-c5b7-465a-9b76-8143abd61e92-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.859318 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.859243    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2f29701-c5b7-465a-9b76-8143abd61e92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.860061 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.860033    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2f29701-c5b7-465a-9b76-8143abd61e92-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.861345 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.861318    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2f29701-c5b7-465a-9b76-8143abd61e92-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.861438 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.861344    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-web-config\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.861498 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.861450    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.861548 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.861494    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.861548 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.861526    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.861723 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.861702    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.862144 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.862125    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-config-volume\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.862364 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.862342    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2f29701-c5b7-465a-9b76-8143abd61e92-config-out\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.863370 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.863350    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d2f29701-c5b7-465a-9b76-8143abd61e92-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.867221 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.867200    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvzgf\" (UniqueName: \"kubernetes.io/projected/d2f29701-c5b7-465a-9b76-8143abd61e92-kube-api-access-mvzgf\") pod \"alertmanager-main-0\" (UID: \"d2f29701-c5b7-465a-9b76-8143abd61e92\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:02.957127 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:02.957092    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 14:37:03.254634 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:03.254609    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 14:37:03.257103 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:37:03.257075    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f29701_c5b7_465a_9b76_8143abd61e92.slice/crio-01b86100f8cd02f6f934e808de81c0e92421c7849499e8b3f61e8662ad5ada6c WatchSource:0}: Error finding container 01b86100f8cd02f6f934e808de81c0e92421c7849499e8b3f61e8662ad5ada6c: Status 404 returned error can't find the container with id 01b86100f8cd02f6f934e808de81c0e92421c7849499e8b3f61e8662ad5ada6c
Sep 08 14:37:03.580392 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:03.580299    2583 generic.go:356] "Generic (PLEG): container finished" podID="d2f29701-c5b7-465a-9b76-8143abd61e92" containerID="764cbbbab879c3ff6140c752e824382c300385b57b85179d7a0079aa1722a6c3" exitCode=0
Sep 08 14:37:03.580724 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:03.580387    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d2f29701-c5b7-465a-9b76-8143abd61e92","Type":"ContainerDied","Data":"764cbbbab879c3ff6140c752e824382c300385b57b85179d7a0079aa1722a6c3"}
Sep 08 14:37:03.580724 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:03.580459    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d2f29701-c5b7-465a-9b76-8143abd61e92","Type":"ContainerStarted","Data":"01b86100f8cd02f6f934e808de81c0e92421c7849499e8b3f61e8662ad5ada6c"}
Sep 08 14:37:03.793252 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:03.793230    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="241686e6-f732-43d5-b6fc-d9eb922215c9" path="/var/lib/kubelet/pods/241686e6-f732-43d5-b6fc-d9eb922215c9/volumes"
Sep 08 14:37:04.592350 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:04.592313    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d2f29701-c5b7-465a-9b76-8143abd61e92","Type":"ContainerStarted","Data":"e57aa41b4ad5900d5be56983a647ac6c5dca661da8c22242351b5e34371bd2ce"}
Sep 08 14:37:04.592785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:04.592415    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d2f29701-c5b7-465a-9b76-8143abd61e92","Type":"ContainerStarted","Data":"327bbf33ac10bc4edd7cc57c7815db022d11a2a45067d4aa6f4effb4775e00de"}
Sep 08 14:37:04.592785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:04.592460    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d2f29701-c5b7-465a-9b76-8143abd61e92","Type":"ContainerStarted","Data":"de7551c24306b0dbe89cc5130214386425797672fec13b05bfb55fcb8c4471ff"}
Sep 08 14:37:04.592785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:04.592502    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d2f29701-c5b7-465a-9b76-8143abd61e92","Type":"ContainerStarted","Data":"296188ab4cdd71f713f05ee647772f18d601b9d5b5aeebb6f96db12998e26b7f"}
Sep 08 14:37:04.592785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:04.592539    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d2f29701-c5b7-465a-9b76-8143abd61e92","Type":"ContainerStarted","Data":"3a3fcdfe2a1e4a14e59e35c3dd1d167d14b94de310541b5687a2443c844f03f8"}
Sep 08 14:37:04.592785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:04.592577    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d2f29701-c5b7-465a-9b76-8143abd61e92","Type":"ContainerStarted","Data":"ef11baf21c4b8258a85e184e5bd41b53a88e7f97d67c0cfc6b77095c9c0120fb"}
Sep 08 14:37:04.621190 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:04.621143    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.602097818 podStartE2EDuration="2.621124436s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.026618ms" isStatefulPod=false podCreationTimestamp="2026-09-08 14:37:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:37:04.618791481 +0000 UTC m=+175.346608494" watchObservedRunningTime="2026-09-08 14:37:04.621124436 +0000 UTC m=+175.348941449"
Sep 08 14:37:08.815896 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:08.815857    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:08.875692 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:08.875663    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:18.306640 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:18.306603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:18.407744 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:18.407707    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:23.823132 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:23.823095    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cb87dcf7c-tn4d9"]
Sep 08 14:37:23.826488 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:23.826464    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:23.838115 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:23.838090    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cb87dcf7c-tn4d9"]
Sep 08 14:37:23.896186 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:23.896152    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:23.919306 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:23.919276    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-oauth-config\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:23.919425 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:23.919312    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-service-ca\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:23.919425 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:23.919332    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-console-config\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:23.919425 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:23.919348    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-trusted-ca-bundle\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:23.919425 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:23.919414    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-serving-cert\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:23.919576 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:23.919433    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5ww8\" (UniqueName: \"kubernetes.io/projected/5094fd28-99da-45c7-b98d-3634c9e838ee-kube-api-access-d5ww8\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:23.919576 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:23.919454    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-oauth-serving-cert\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.019960 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.019927    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-oauth-config\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.020119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.019965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-service-ca\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.020119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.019985    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-console-config\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.020119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.019999    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-trusted-ca-bundle\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.020119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.020030    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-serving-cert\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.020119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.020057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5ww8\" (UniqueName: \"kubernetes.io/projected/5094fd28-99da-45c7-b98d-3634c9e838ee-kube-api-access-d5ww8\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.020119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.020087    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-oauth-serving-cert\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.020902 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.020874    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-service-ca\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.021039 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.020925    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-console-config\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.021149 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.020925    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-oauth-serving-cert\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.021149 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.020972    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-trusted-ca-bundle\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.022711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.022689    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-oauth-config\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.022767 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.022737    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-serving-cert\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.029825 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.029800    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5ww8\" (UniqueName: \"kubernetes.io/projected/5094fd28-99da-45c7-b98d-3634c9e838ee-kube-api-access-d5ww8\") pod \"console-7cb87dcf7c-tn4d9\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") " pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.136096 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.136032    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:24.407629 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.407560    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cb87dcf7c-tn4d9"]
Sep 08 14:37:24.410960 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:37:24.410935    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5094fd28_99da_45c7_b98d_3634c9e838ee.slice/crio-698ac568b4dc469df3e77e6adc7422723586babf279f7020a524d367ff4b166d WatchSource:0}: Error finding container 698ac568b4dc469df3e77e6adc7422723586babf279f7020a524d367ff4b166d: Status 404 returned error can't find the container with id 698ac568b4dc469df3e77e6adc7422723586babf279f7020a524d367ff4b166d
Sep 08 14:37:24.673354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.673265    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb87dcf7c-tn4d9" event={"ID":"5094fd28-99da-45c7-b98d-3634c9e838ee","Type":"ContainerStarted","Data":"76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a"}
Sep 08 14:37:24.673354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.673312    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb87dcf7c-tn4d9" event={"ID":"5094fd28-99da-45c7-b98d-3634c9e838ee","Type":"ContainerStarted","Data":"698ac568b4dc469df3e77e6adc7422723586babf279f7020a524d367ff4b166d"}
Sep 08 14:37:24.693379 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:24.693322    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cb87dcf7c-tn4d9" podStartSLOduration=1.693306161 podStartE2EDuration="1.693306161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:37:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:37:24.692343149 +0000 UTC m=+195.420160157" watchObservedRunningTime="2026-09-08 14:37:24.693306161 +0000 UTC m=+195.421123230"
Sep 08 14:37:34.136756 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:34.136706    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:34.137278 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:34.136782    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:34.141399 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:34.141377    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:34.720841 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:34.720798    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:37:34.766922 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:34.766888    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59d5974b85-kdfhj"]
Sep 08 14:37:34.888740 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:34.888707    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:34.946683 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:34.946654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:35.003659 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:35.003576    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:37:59.793779 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:37:59.793744    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59d5974b85-kdfhj" podUID="6c769e47-42d4-4d29-bdc0-0c062d1df5ea" containerName="console" containerID="cri-o://b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a" gracePeriod=15
Sep 08 14:38:00.031575 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.031554    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:38:00.108525 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.108461    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-serving-cert\") pod \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") "
Sep 08 14:38:00.108525 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.108494    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-oauth-config\") pod \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") "
Sep 08 14:38:00.108525 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.108519    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-oauth-serving-cert\") pod \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") "
Sep 08 14:38:00.108767 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.108537    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-service-ca\") pod \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") "
Sep 08 14:38:00.108767 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.108600    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gklmq\" (UniqueName: \"kubernetes.io/projected/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-kube-api-access-gklmq\") pod \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") "
Sep 08 14:38:00.108767 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.108632    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-config\") pod \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") "
Sep 08 14:38:00.108767 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.108670    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-trusted-ca-bundle\") pod \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\" (UID: \"6c769e47-42d4-4d29-bdc0-0c062d1df5ea\") "
Sep 08 14:38:00.109000 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.108970    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-service-ca" (OuterVolumeSpecName: "service-ca") pod "6c769e47-42d4-4d29-bdc0-0c062d1df5ea" (UID: "6c769e47-42d4-4d29-bdc0-0c062d1df5ea"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:38:00.109000 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.108979    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6c769e47-42d4-4d29-bdc0-0c062d1df5ea" (UID: "6c769e47-42d4-4d29-bdc0-0c062d1df5ea"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:38:00.109089 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.109014    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-config" (OuterVolumeSpecName: "console-config") pod "6c769e47-42d4-4d29-bdc0-0c062d1df5ea" (UID: "6c769e47-42d4-4d29-bdc0-0c062d1df5ea"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:38:00.109246 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.109228    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6c769e47-42d4-4d29-bdc0-0c062d1df5ea" (UID: "6c769e47-42d4-4d29-bdc0-0c062d1df5ea"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:38:00.110814 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.110786    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6c769e47-42d4-4d29-bdc0-0c062d1df5ea" (UID: "6c769e47-42d4-4d29-bdc0-0c062d1df5ea"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:38:00.111296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.111271    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6c769e47-42d4-4d29-bdc0-0c062d1df5ea" (UID: "6c769e47-42d4-4d29-bdc0-0c062d1df5ea"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:38:00.111387 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.111293    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-kube-api-access-gklmq" (OuterVolumeSpecName: "kube-api-access-gklmq") pod "6c769e47-42d4-4d29-bdc0-0c062d1df5ea" (UID: "6c769e47-42d4-4d29-bdc0-0c062d1df5ea"). InnerVolumeSpecName "kube-api-access-gklmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:38:00.210274 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.210250    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-serving-cert\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:38:00.210274 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.210272    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-oauth-config\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:38:00.210401 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.210281    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-oauth-serving-cert\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:38:00.210401 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.210290    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:38:00.210401 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.210300    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gklmq\" (UniqueName: \"kubernetes.io/projected/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-kube-api-access-gklmq\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:38:00.210401 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.210308    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-console-config\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:38:00.210401 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.210316    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c769e47-42d4-4d29-bdc0-0c062d1df5ea-trusted-ca-bundle\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:38:00.818355 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.818321    2583 generic.go:356] "Generic (PLEG): container finished" podID="6c769e47-42d4-4d29-bdc0-0c062d1df5ea" containerID="b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a" exitCode=0
Sep 08 14:38:00.818752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.818373    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d5974b85-kdfhj"
Sep 08 14:38:00.818752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.818379    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d5974b85-kdfhj" event={"ID":"6c769e47-42d4-4d29-bdc0-0c062d1df5ea","Type":"ContainerDied","Data":"b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a"}
Sep 08 14:38:00.818752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.818414    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d5974b85-kdfhj" event={"ID":"6c769e47-42d4-4d29-bdc0-0c062d1df5ea","Type":"ContainerDied","Data":"753c7570e6db785375b1c1c816a3b04ce42ef060c80964dd6ba42c9a41ce936c"}
Sep 08 14:38:00.818752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.818432    2583 scope.go:117] "RemoveContainer" containerID="b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a"
Sep 08 14:38:00.827145 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.827127    2583 scope.go:117] "RemoveContainer" containerID="b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a"
Sep 08 14:38:00.827399 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:38:00.827374    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a\": container with ID starting with b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a not found: ID does not exist" containerID="b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a"
Sep 08 14:38:00.827443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.827409    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a"} err="failed to get container status \"b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a\": rpc error: code = NotFound desc = could not find container \"b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a\": container with ID starting with b8d806701a6db1a91cb3716071901c9be6591a4a68745273ebfbd700c69e157a not found: ID does not exist"
Sep 08 14:38:00.839607 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.839580    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59d5974b85-kdfhj"]
Sep 08 14:38:00.843276 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:00.843251    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59d5974b85-kdfhj"]
Sep 08 14:38:01.792115 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:01.792077    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c769e47-42d4-4d29-bdc0-0c062d1df5ea" path="/var/lib/kubelet/pods/6c769e47-42d4-4d29-bdc0-0c062d1df5ea/volumes"
Sep 08 14:38:01.939574 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:01.939544    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:02.006354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:02.006324    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:05.842653 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:05.842612    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:05.936245 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:05.936218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:14.584814 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:14.584769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:27.750316 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:27.750272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:29.133896 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:29.133849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:31.367486 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:31.367396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:31.469116 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:31.469079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:33.381072 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.381046    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:33.474413 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.474383    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:33.598253 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.598210    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9mhqm"]
Sep 08 14:38:33.598525 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.598512    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c769e47-42d4-4d29-bdc0-0c062d1df5ea" containerName="console"
Sep 08 14:38:33.598567 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.598527    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c769e47-42d4-4d29-bdc0-0c062d1df5ea" containerName="console"
Sep 08 14:38:33.598601 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.598588    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c769e47-42d4-4d29-bdc0-0c062d1df5ea" containerName="console"
Sep 08 14:38:33.600576 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.600561    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm"
Sep 08 14:38:33.605218 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.605191    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-5ftlx\""
Sep 08 14:38:33.606376 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.606356    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 08 14:38:33.606730 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.606707    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 08 14:38:33.616427 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.616405    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9mhqm"]
Sep 08 14:38:33.682942 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.682855    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t5h7\" (UniqueName: \"kubernetes.io/projected/1807561f-a5d3-4892-abdb-4a25e487bf16-kube-api-access-4t5h7\") pod \"cert-manager-webhook-6486ffd997-9mhqm\" (UID: \"1807561f-a5d3-4892-abdb-4a25e487bf16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm"
Sep 08 14:38:33.683086 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.683006    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1807561f-a5d3-4892-abdb-4a25e487bf16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9mhqm\" (UID: \"1807561f-a5d3-4892-abdb-4a25e487bf16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm"
Sep 08 14:38:33.783422 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.783390    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1807561f-a5d3-4892-abdb-4a25e487bf16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9mhqm\" (UID: \"1807561f-a5d3-4892-abdb-4a25e487bf16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm"
Sep 08 14:38:33.783591 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.783446    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t5h7\" (UniqueName: \"kubernetes.io/projected/1807561f-a5d3-4892-abdb-4a25e487bf16-kube-api-access-4t5h7\") pod \"cert-manager-webhook-6486ffd997-9mhqm\" (UID: \"1807561f-a5d3-4892-abdb-4a25e487bf16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm"
Sep 08 14:38:33.793303 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.793277    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t5h7\" (UniqueName: \"kubernetes.io/projected/1807561f-a5d3-4892-abdb-4a25e487bf16-kube-api-access-4t5h7\") pod \"cert-manager-webhook-6486ffd997-9mhqm\" (UID: \"1807561f-a5d3-4892-abdb-4a25e487bf16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm"
Sep 08 14:38:33.794803 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.794781    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1807561f-a5d3-4892-abdb-4a25e487bf16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9mhqm\" (UID: \"1807561f-a5d3-4892-abdb-4a25e487bf16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm"
Sep 08 14:38:33.910086 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:33.910058    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm"
Sep 08 14:38:34.082534 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:34.082498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:34.189341 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:34.189309    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:34.197462 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:34.197441    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9mhqm"]
Sep 08 14:38:34.200012 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:38:34.199981    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1807561f_a5d3_4892_abdb_4a25e487bf16.slice/crio-0f442a54c1350cda6b6b509b6c37f6423993d372007e38ac364c055dae97295b WatchSource:0}: Error finding container 0f442a54c1350cda6b6b509b6c37f6423993d372007e38ac364c055dae97295b: Status 404 returned error can't find the container with id 0f442a54c1350cda6b6b509b6c37f6423993d372007e38ac364c055dae97295b
Sep 08 14:38:34.943578 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:34.943542    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm" event={"ID":"1807561f-a5d3-4892-abdb-4a25e487bf16","Type":"ContainerStarted","Data":"0f442a54c1350cda6b6b509b6c37f6423993d372007e38ac364c055dae97295b"}
Sep 08 14:38:35.617389 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:35.617351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:35.805869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:35.805812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:36.386340 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:36.386304    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-57fqm"]
Sep 08 14:38:36.390191 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:36.390169    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-57fqm"
Sep 08 14:38:36.392886 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:36.392863    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-pfpcb\""
Sep 08 14:38:36.396269 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:36.396246    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-57fqm"]
Sep 08 14:38:36.508290 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:36.508255    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c51afef-32c5-44c0-929f-af1fbd5939d6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-57fqm\" (UID: \"6c51afef-32c5-44c0-929f-af1fbd5939d6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-57fqm"
Sep 08 14:38:36.508478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:36.508307    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s872x\" (UniqueName: \"kubernetes.io/projected/6c51afef-32c5-44c0-929f-af1fbd5939d6-kube-api-access-s872x\") pod \"cert-manager-cainjector-559cdd8c87-57fqm\" (UID: \"6c51afef-32c5-44c0-929f-af1fbd5939d6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-57fqm"
Sep 08 14:38:36.609771 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:36.609729    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s872x\" (UniqueName: \"kubernetes.io/projected/6c51afef-32c5-44c0-929f-af1fbd5939d6-kube-api-access-s872x\") pod \"cert-manager-cainjector-559cdd8c87-57fqm\" (UID: \"6c51afef-32c5-44c0-929f-af1fbd5939d6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-57fqm"
Sep 08 14:38:36.610049 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:36.609853    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c51afef-32c5-44c0-929f-af1fbd5939d6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-57fqm\" (UID: \"6c51afef-32c5-44c0-929f-af1fbd5939d6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-57fqm"
Sep 08 14:38:36.619427 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:36.619386    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c51afef-32c5-44c0-929f-af1fbd5939d6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-57fqm\" (UID: \"6c51afef-32c5-44c0-929f-af1fbd5939d6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-57fqm"
Sep 08 14:38:36.619427 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:36.619393    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s872x\" (UniqueName: \"kubernetes.io/projected/6c51afef-32c5-44c0-929f-af1fbd5939d6-kube-api-access-s872x\") pod \"cert-manager-cainjector-559cdd8c87-57fqm\" (UID: \"6c51afef-32c5-44c0-929f-af1fbd5939d6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-57fqm"
Sep 08 14:38:36.702451 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:36.702372    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-57fqm"
Sep 08 14:38:37.677202 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:37.677179    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-57fqm"]
Sep 08 14:38:37.679443 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:38:37.679416    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c51afef_32c5_44c0_929f_af1fbd5939d6.slice/crio-763b640c57d10aa774a82c6c9014558dfe7f13529b68843ce69dea638ff7cc66 WatchSource:0}: Error finding container 763b640c57d10aa774a82c6c9014558dfe7f13529b68843ce69dea638ff7cc66: Status 404 returned error can't find the container with id 763b640c57d10aa774a82c6c9014558dfe7f13529b68843ce69dea638ff7cc66
Sep 08 14:38:37.964279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:37.964187    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm" event={"ID":"1807561f-a5d3-4892-abdb-4a25e487bf16","Type":"ContainerStarted","Data":"92cfb862ff08df924bc47b76ad2cb3362f017e21ec66667fe27d3c2334f3506a"}
Sep 08 14:38:37.964279 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:37.964237    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm"
Sep 08 14:38:37.965892 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:37.965864    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-57fqm" event={"ID":"6c51afef-32c5-44c0-929f-af1fbd5939d6","Type":"ContainerStarted","Data":"65b0ae46e460c8e107e4abb531ab0a8551ef1289ff2a788a6795c919a5dc7936"}
Sep 08 14:38:37.966008 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:37.965904    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-57fqm" event={"ID":"6c51afef-32c5-44c0-929f-af1fbd5939d6","Type":"ContainerStarted","Data":"763b640c57d10aa774a82c6c9014558dfe7f13529b68843ce69dea638ff7cc66"}
Sep 08 14:38:37.999559 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:37.999504    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm" podStartSLOduration=1.7466958959999999 podStartE2EDuration="4.999490766s" totalImagesPullingTime="3.25279487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:38:37.981879498 +0000 UTC m=+268.709696513" watchObservedRunningTime="2026-09-08 14:38:37.999490766 +0000 UTC m=+268.727307819"
Sep 08 14:38:37.999708 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:37.999594    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-57fqm" podStartSLOduration=1.999587432 podStartE2EDuration="1.999587432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:38:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:38:37.998783769 +0000 UTC m=+268.726600783" watchObservedRunningTime="2026-09-08 14:38:37.999587432 +0000 UTC m=+268.727404446"
Sep 08 14:38:38.153387 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:38.153360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:38.215155 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:38.215087    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:43.972566 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:43.972536    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9mhqm"
Sep 08 14:38:43.981161 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:43.981136    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:45.290872 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.290823    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mcw5r"]
Sep 08 14:38:45.293277 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.293255    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mcw5r"
Sep 08 14:38:45.296013 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.295992    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hlbvs\""
Sep 08 14:38:45.302410 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.302391    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mcw5r"]
Sep 08 14:38:45.483974 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.483940    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7k6w\" (UniqueName: \"kubernetes.io/projected/7714c708-9774-417c-bb8c-1bdad90ba4cd-kube-api-access-z7k6w\") pod \"cert-manager-55b4d954d5-mcw5r\" (UID: \"7714c708-9774-417c-bb8c-1bdad90ba4cd\") " pod="cert-manager/cert-manager-55b4d954d5-mcw5r"
Sep 08 14:38:45.483974 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.483978    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7714c708-9774-417c-bb8c-1bdad90ba4cd-bound-sa-token\") pod \"cert-manager-55b4d954d5-mcw5r\" (UID: \"7714c708-9774-417c-bb8c-1bdad90ba4cd\") " pod="cert-manager/cert-manager-55b4d954d5-mcw5r"
Sep 08 14:38:45.585291 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.585200    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7714c708-9774-417c-bb8c-1bdad90ba4cd-bound-sa-token\") pod \"cert-manager-55b4d954d5-mcw5r\" (UID: \"7714c708-9774-417c-bb8c-1bdad90ba4cd\") " pod="cert-manager/cert-manager-55b4d954d5-mcw5r"
Sep 08 14:38:45.585430 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.585320    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7k6w\" (UniqueName: \"kubernetes.io/projected/7714c708-9774-417c-bb8c-1bdad90ba4cd-kube-api-access-z7k6w\") pod \"cert-manager-55b4d954d5-mcw5r\" (UID: \"7714c708-9774-417c-bb8c-1bdad90ba4cd\") " pod="cert-manager/cert-manager-55b4d954d5-mcw5r"
Sep 08 14:38:45.595395 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.595359    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7714c708-9774-417c-bb8c-1bdad90ba4cd-bound-sa-token\") pod \"cert-manager-55b4d954d5-mcw5r\" (UID: \"7714c708-9774-417c-bb8c-1bdad90ba4cd\") " pod="cert-manager/cert-manager-55b4d954d5-mcw5r"
Sep 08 14:38:45.595509 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.595475    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7k6w\" (UniqueName: \"kubernetes.io/projected/7714c708-9774-417c-bb8c-1bdad90ba4cd-kube-api-access-z7k6w\") pod \"cert-manager-55b4d954d5-mcw5r\" (UID: \"7714c708-9774-417c-bb8c-1bdad90ba4cd\") " pod="cert-manager/cert-manager-55b4d954d5-mcw5r"
Sep 08 14:38:45.602137 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.602116    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mcw5r"
Sep 08 14:38:45.870168 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.870088    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mcw5r"]
Sep 08 14:38:45.872983 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:38:45.872957    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7714c708_9774_417c_bb8c_1bdad90ba4cd.slice/crio-c6deec1e2d30aaf4859624589f9b2e4c51203903d3d73f3dfcc01a9196a2b605 WatchSource:0}: Error finding container c6deec1e2d30aaf4859624589f9b2e4c51203903d3d73f3dfcc01a9196a2b605: Status 404 returned error can't find the container with id c6deec1e2d30aaf4859624589f9b2e4c51203903d3d73f3dfcc01a9196a2b605
Sep 08 14:38:45.998721 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.998686    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mcw5r" event={"ID":"7714c708-9774-417c-bb8c-1bdad90ba4cd","Type":"ContainerStarted","Data":"4301fcce287dbe1d45fd84561a3ce257611ced10f718a9dae099dfdab0ab7751"}
Sep 08 14:38:45.998721 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:45.998727    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mcw5r" event={"ID":"7714c708-9774-417c-bb8c-1bdad90ba4cd","Type":"ContainerStarted","Data":"c6deec1e2d30aaf4859624589f9b2e4c51203903d3d73f3dfcc01a9196a2b605"}
Sep 08 14:38:46.016233 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:46.016187    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-mcw5r" podStartSLOduration=1.016170909 podStartE2EDuration="1.016170909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:38:46.015647945 +0000 UTC m=+276.743464960" watchObservedRunningTime="2026-09-08 14:38:46.016170909 +0000 UTC m=+276.743987922"
Sep 08 14:38:46.648432 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:46.648396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:46.750850 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:46.750801    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:51.440308 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:51.440272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:51.548108 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:51.548076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:53.303050 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:53.302997    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:54.804950 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:54.804916    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:38:54.864570 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:38:54.864534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:05.567430 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:05.567399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:08.402679 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:08.402642    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:08.504453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:08.504409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:08.990315 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:08.990268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:09.214334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.214300    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:09.314867 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.314761    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:09.369515 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.369485    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"]
Sep 08 14:39:09.374820 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.374797    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.380004 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.379979    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 14:39:09.380138 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.380004    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 14:39:09.380138 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.380020    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q8mbk\""
Sep 08 14:39:09.380138 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.380101    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 14:39:09.380293 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.380179    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 14:39:09.392015 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.391991    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"]
Sep 08 14:39:09.468284 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.468256    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp2t8\" (UniqueName: \"kubernetes.io/projected/ae5ee9f8-55b4-4916-8cc4-a705216385e5-kube-api-access-wp2t8\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.468668 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.468300    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae5ee9f8-55b4-4916-8cc4-a705216385e5-manifests\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.468668 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.468404    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae5ee9f8-55b4-4916-8cc4-a705216385e5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.468668 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.468438    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae5ee9f8-55b4-4916-8cc4-a705216385e5-webhook-cert\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.468668 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.468469    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae5ee9f8-55b4-4916-8cc4-a705216385e5-tmp\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.517112 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.517073    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:09.569892 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.569808    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae5ee9f8-55b4-4916-8cc4-a705216385e5-tmp\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.569892 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.569873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp2t8\" (UniqueName: \"kubernetes.io/projected/ae5ee9f8-55b4-4916-8cc4-a705216385e5-kube-api-access-wp2t8\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.570050 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.569904    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae5ee9f8-55b4-4916-8cc4-a705216385e5-manifests\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.570050 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.569979    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae5ee9f8-55b4-4916-8cc4-a705216385e5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.570050 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.569996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae5ee9f8-55b4-4916-8cc4-a705216385e5-webhook-cert\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.570386 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.570367    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae5ee9f8-55b4-4916-8cc4-a705216385e5-manifests\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.572304 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.572285    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae5ee9f8-55b4-4916-8cc4-a705216385e5-tmp\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.572652 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.572632    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae5ee9f8-55b4-4916-8cc4-a705216385e5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.572696 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.572631    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae5ee9f8-55b4-4916-8cc4-a705216385e5-webhook-cert\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.578967 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.578942    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp2t8\" (UniqueName: \"kubernetes.io/projected/ae5ee9f8-55b4-4916-8cc4-a705216385e5-kube-api-access-wp2t8\") pod \"opendatahub-operator-controller-manager-766f77f8db-fdkgr\" (UID: \"ae5ee9f8-55b4-4916-8cc4-a705216385e5\") " pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.620493 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.620468    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:09.676631 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.676595    2583 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 14:39:09.687867 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.687847    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q8mbk\""
Sep 08 14:39:09.696200 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.696183    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:09.726629 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.726596    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:09.819008 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.818979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:09.920541 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.920451    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:09.980528 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:09.980499    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"]
Sep 08 14:39:09.984355 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:39:09.984328    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5ee9f8_55b4_4916_8cc4_a705216385e5.slice/crio-4b80f592413ae4276864af52b28e4688971619e40ce58f09abbaff8f8b71fa51 WatchSource:0}: Error finding container 4b80f592413ae4276864af52b28e4688971619e40ce58f09abbaff8f8b71fa51: Status 404 returned error can't find the container with id 4b80f592413ae4276864af52b28e4688971619e40ce58f09abbaff8f8b71fa51
Sep 08 14:39:10.018744 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:10.018714    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:10.094856 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:10.094808    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr" event={"ID":"ae5ee9f8-55b4-4916-8cc4-a705216385e5","Type":"ContainerStarted","Data":"4b80f592413ae4276864af52b28e4688971619e40ce58f09abbaff8f8b71fa51"}
Sep 08 14:39:11.628861 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:11.628814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:11.719422 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:11.719385    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:14.116494 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:14.116460    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr" event={"ID":"ae5ee9f8-55b4-4916-8cc4-a705216385e5","Type":"ContainerStarted","Data":"00eec0a21885476df296c99d1332bcd7a2d8097b85964f89d2e9f66dc2a91b9f"}
Sep 08 14:39:15.122069 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:15.122034    2583 generic.go:356] "Generic (PLEG): container finished" podID="ae5ee9f8-55b4-4916-8cc4-a705216385e5" containerID="00eec0a21885476df296c99d1332bcd7a2d8097b85964f89d2e9f66dc2a91b9f" exitCode=0
Sep 08 14:39:15.122486 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:15.122126    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr" event={"ID":"ae5ee9f8-55b4-4916-8cc4-a705216385e5","Type":"ContainerDied","Data":"00eec0a21885476df296c99d1332bcd7a2d8097b85964f89d2e9f66dc2a91b9f"}
Sep 08 14:39:15.122548 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:15.122532    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:39:16.128619 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:16.128579    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr" event={"ID":"ae5ee9f8-55b4-4916-8cc4-a705216385e5","Type":"ContainerStarted","Data":"bdf06b0d07366519a2a8b0c94c9e517941300b5840879e0641e011917f01976e"}
Sep 08 14:39:16.129025 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:16.128703    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:16.156495 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:16.156443    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr" podStartSLOduration=2.46949862 podStartE2EDuration="7.15642959s" totalImagesPullingTime="3.661389357s" totalInitContainerRuntime="1.025541613s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:39:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:39:16.153720479 +0000 UTC m=+306.881537493" watchObservedRunningTime="2026-09-08 14:39:16.15642959 +0000 UTC m=+306.884246667"
Sep 08 14:39:26.840861 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:26.840799    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:27.135166 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:27.135087    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-766f77f8db-fdkgr"
Sep 08 14:39:28.651854 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:28.651799    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:28.796569 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:28.796538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:29.204645 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.204615    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:29.222909 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.222881    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"]
Sep 08 14:39:29.228306 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.228288    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"
Sep 08 14:39:29.239703 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.239681    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 08 14:39:29.242804 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.242786    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 08 14:39:29.244588 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.244572    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-c6tv2\""
Sep 08 14:39:29.251488 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.251462    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"]
Sep 08 14:39:29.313398 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.313370    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:29.335499 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.335471    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91ead5be-c3b9-4f61-931f-75199eeeae6f-tls-certs\") pod \"kube-auth-proxy-6dd8bb6964-pskvx\" (UID: \"91ead5be-c3b9-4f61-931f-75199eeeae6f\") " pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"
Sep 08 14:39:29.335614 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.335542    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91ead5be-c3b9-4f61-931f-75199eeeae6f-tmp\") pod \"kube-auth-proxy-6dd8bb6964-pskvx\" (UID: \"91ead5be-c3b9-4f61-931f-75199eeeae6f\") " pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"
Sep 08 14:39:29.335614 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.335577    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtkv2\" (UniqueName: \"kubernetes.io/projected/91ead5be-c3b9-4f61-931f-75199eeeae6f-kube-api-access-xtkv2\") pod \"kube-auth-proxy-6dd8bb6964-pskvx\" (UID: \"91ead5be-c3b9-4f61-931f-75199eeeae6f\") " pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"
Sep 08 14:39:29.436794 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.436750    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91ead5be-c3b9-4f61-931f-75199eeeae6f-tmp\") pod \"kube-auth-proxy-6dd8bb6964-pskvx\" (UID: \"91ead5be-c3b9-4f61-931f-75199eeeae6f\") " pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"
Sep 08 14:39:29.436794 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.436803    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtkv2\" (UniqueName: \"kubernetes.io/projected/91ead5be-c3b9-4f61-931f-75199eeeae6f-kube-api-access-xtkv2\") pod \"kube-auth-proxy-6dd8bb6964-pskvx\" (UID: \"91ead5be-c3b9-4f61-931f-75199eeeae6f\") " pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"
Sep 08 14:39:29.437019 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.436880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91ead5be-c3b9-4f61-931f-75199eeeae6f-tls-certs\") pod \"kube-auth-proxy-6dd8bb6964-pskvx\" (UID: \"91ead5be-c3b9-4f61-931f-75199eeeae6f\") " pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"
Sep 08 14:39:29.439289 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.439266    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91ead5be-c3b9-4f61-931f-75199eeeae6f-tmp\") pod \"kube-auth-proxy-6dd8bb6964-pskvx\" (UID: \"91ead5be-c3b9-4f61-931f-75199eeeae6f\") " pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"
Sep 08 14:39:29.439417 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.439400    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91ead5be-c3b9-4f61-931f-75199eeeae6f-tls-certs\") pod \"kube-auth-proxy-6dd8bb6964-pskvx\" (UID: \"91ead5be-c3b9-4f61-931f-75199eeeae6f\") " pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"
Sep 08 14:39:29.445733 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.445713    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtkv2\" (UniqueName: \"kubernetes.io/projected/91ead5be-c3b9-4f61-931f-75199eeeae6f-kube-api-access-xtkv2\") pod \"kube-auth-proxy-6dd8bb6964-pskvx\" (UID: \"91ead5be-c3b9-4f61-931f-75199eeeae6f\") " pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"
Sep 08 14:39:29.537897 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.537871    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"
Sep 08 14:39:29.808040 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:29.808017    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx"]
Sep 08 14:39:29.810080 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:39:29.810052    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ead5be_c3b9_4f61_931f_75199eeeae6f.slice/crio-88c5c88c878ed1430959d19772da07d08c96bf82becbc6ff5a29ec34354c8e54 WatchSource:0}: Error finding container 88c5c88c878ed1430959d19772da07d08c96bf82becbc6ff5a29ec34354c8e54: Status 404 returned error can't find the container with id 88c5c88c878ed1430959d19772da07d08c96bf82becbc6ff5a29ec34354c8e54
Sep 08 14:39:30.197239 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:30.197133    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx" event={"ID":"91ead5be-c3b9-4f61-931f-75199eeeae6f","Type":"ContainerStarted","Data":"88c5c88c878ed1430959d19772da07d08c96bf82becbc6ff5a29ec34354c8e54"}
Sep 08 14:39:30.664057 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:30.664020    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:30.765166 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:30.765118    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:30.854209 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:30.854116    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:31.887755 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:31.887720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:33.212735 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:33.212699    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx" event={"ID":"91ead5be-c3b9-4f61-931f-75199eeeae6f","Type":"ContainerStarted","Data":"403e0c0660439f1aacbceaf340b39841526dff751873d05cf9c68d2d27943875"}
Sep 08 14:39:33.232050 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:33.231999    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6dd8bb6964-pskvx" podStartSLOduration=1.5547040060000001 podStartE2EDuration="4.231980039s" totalImagesPullingTime="2.677276033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:39:33.230593229 +0000 UTC m=+323.958410243" watchObservedRunningTime="2026-09-08 14:39:33.231980039 +0000 UTC m=+323.959797044"
Sep 08 14:39:36.957202 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:36.957172    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:37.050195 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:37.050163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:37.616238 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:37.616204    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:37.767748 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:37.767706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:37.868455 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:37.868363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:39.844525 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:39.844461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:39.942115 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:39.942079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:40.005474 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.005440    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:40.358065 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.358037    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9pf4x"]
Sep 08 14:39:40.363917 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.362193    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x"
Sep 08 14:39:40.365343 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.365320    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 08 14:39:40.372407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.372382    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9pf4x"]
Sep 08 14:39:40.452132 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.452098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:40.532874 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.532826    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c91d268d-4b1a-4d68-ad3e-fc89ba9f4318-tls-certs\") pod \"model-serving-api-54d89bd5fc-9pf4x\" (UID: \"c91d268d-4b1a-4d68-ad3e-fc89ba9f4318\") " pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x"
Sep 08 14:39:40.533026 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.532881    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smmrh\" (UniqueName: \"kubernetes.io/projected/c91d268d-4b1a-4d68-ad3e-fc89ba9f4318-kube-api-access-smmrh\") pod \"model-serving-api-54d89bd5fc-9pf4x\" (UID: \"c91d268d-4b1a-4d68-ad3e-fc89ba9f4318\") " pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x"
Sep 08 14:39:40.633783 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.633711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smmrh\" (UniqueName: \"kubernetes.io/projected/c91d268d-4b1a-4d68-ad3e-fc89ba9f4318-kube-api-access-smmrh\") pod \"model-serving-api-54d89bd5fc-9pf4x\" (UID: \"c91d268d-4b1a-4d68-ad3e-fc89ba9f4318\") " pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x"
Sep 08 14:39:40.633931 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.633798    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c91d268d-4b1a-4d68-ad3e-fc89ba9f4318-tls-certs\") pod \"model-serving-api-54d89bd5fc-9pf4x\" (UID: \"c91d268d-4b1a-4d68-ad3e-fc89ba9f4318\") " pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x"
Sep 08 14:39:40.636275 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.636257    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c91d268d-4b1a-4d68-ad3e-fc89ba9f4318-tls-certs\") pod \"model-serving-api-54d89bd5fc-9pf4x\" (UID: \"c91d268d-4b1a-4d68-ad3e-fc89ba9f4318\") " pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x"
Sep 08 14:39:40.642957 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.642939    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smmrh\" (UniqueName: \"kubernetes.io/projected/c91d268d-4b1a-4d68-ad3e-fc89ba9f4318-kube-api-access-smmrh\") pod \"model-serving-api-54d89bd5fc-9pf4x\" (UID: \"c91d268d-4b1a-4d68-ad3e-fc89ba9f4318\") " pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x"
Sep 08 14:39:40.674923 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.674894    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x"
Sep 08 14:39:40.687344 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.687190    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:40.948099 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:40.948073    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9pf4x"]
Sep 08 14:39:40.950635 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:39:40.950612    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91d268d_4b1a_4d68_ad3e_fc89ba9f4318.slice/crio-d3ded3f3388348868778b44ebc7bfbf652bb88e1a878e9233b6afe8ab748efcf WatchSource:0}: Error finding container d3ded3f3388348868778b44ebc7bfbf652bb88e1a878e9233b6afe8ab748efcf: Status 404 returned error can't find the container with id d3ded3f3388348868778b44ebc7bfbf652bb88e1a878e9233b6afe8ab748efcf
Sep 08 14:39:41.246143 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:41.246109    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x" event={"ID":"c91d268d-4b1a-4d68-ad3e-fc89ba9f4318","Type":"ContainerStarted","Data":"d3ded3f3388348868778b44ebc7bfbf652bb88e1a878e9233b6afe8ab748efcf"}
Sep 08 14:39:43.256212 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:43.256183    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x" event={"ID":"c91d268d-4b1a-4d68-ad3e-fc89ba9f4318","Type":"ContainerStarted","Data":"f64f550219e922e5ceee9d7212e5be4530f7773497fc5b4ddfd8e947d985e826"}
Sep 08 14:39:43.256565 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:43.256315    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x"
Sep 08 14:39:43.274712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:43.274658    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x" podStartSLOduration=1.815864816 podStartE2EDuration="3.274642153s" totalImagesPullingTime="1.458777337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:39:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:39:43.272873576 +0000 UTC m=+334.000690594" watchObservedRunningTime="2026-09-08 14:39:43.274642153 +0000 UTC m=+334.002459166"
Sep 08 14:39:46.803183 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.803148    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"]
Sep 08 14:39:46.809879 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.809853    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:46.813122 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.813084    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 08 14:39:46.813122 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.813100    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 08 14:39:46.813338 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.813178    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lq5wr\""
Sep 08 14:39:46.814507 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.814484    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 08 14:39:46.814658 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.814586    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 08 14:39:46.814658 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.814605    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 08 14:39:46.816271 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.816252    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"]
Sep 08 14:39:46.987975 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.987935    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4mjs\" (UniqueName: \"kubernetes.io/projected/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-kube-api-access-k4mjs\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:46.987975 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.987988    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-metrics-cert\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:46.988193 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.988075    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-cert\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:46.988193 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:46.988116    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-manager-config\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:47.089333 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:47.089259    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-cert\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:47.089333 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:47.089294    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-manager-config\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:47.089333 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:47.089329    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4mjs\" (UniqueName: \"kubernetes.io/projected/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-kube-api-access-k4mjs\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:47.089532 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:47.089365    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-metrics-cert\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:47.090039 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:47.089984    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-manager-config\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:47.091711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:47.091681    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-metrics-cert\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:47.091804 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:47.091771    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-cert\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:47.111254 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:47.111233    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4mjs\" (UniqueName: \"kubernetes.io/projected/eba0b560-b8e8-4b80-bddf-dcc9896dcae8-kube-api-access-k4mjs\") pod \"lws-controller-manager-84fb6884f5-xddmw\" (UID: \"eba0b560-b8e8-4b80-bddf-dcc9896dcae8\") " pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:47.120133 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:47.120113    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:47.399407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:47.399205    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"]
Sep 08 14:39:47.402089 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:39:47.402059    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba0b560_b8e8_4b80_bddf_dcc9896dcae8.slice/crio-9aa6fc69a55dcb266a3e4a89bacb002506918963d3074194882e43aac4c97009 WatchSource:0}: Error finding container 9aa6fc69a55dcb266a3e4a89bacb002506918963d3074194882e43aac4c97009: Status 404 returned error can't find the container with id 9aa6fc69a55dcb266a3e4a89bacb002506918963d3074194882e43aac4c97009
Sep 08 14:39:48.279977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:48.279935    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw" event={"ID":"eba0b560-b8e8-4b80-bddf-dcc9896dcae8","Type":"ContainerStarted","Data":"9aa6fc69a55dcb266a3e4a89bacb002506918963d3074194882e43aac4c97009"}
Sep 08 14:39:50.801212 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:50.801174    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:50.907427 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:50.907393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:51.037275 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:51.037218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:51.135426 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:51.135343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:52.116721 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:52.116679    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:52.214711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:52.214655    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:54.264696 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:54.264666    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9pf4x"
Sep 08 14:39:55.155730 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:55.155699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:55.256379 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:55.256348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:39:58.326455 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:58.326416    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw" event={"ID":"eba0b560-b8e8-4b80-bddf-dcc9896dcae8","Type":"ContainerStarted","Data":"f68e3913149de61709dd3c405de3a3b4f5efbff095bd660d097f53abd91092ae"}
Sep 08 14:39:58.326823 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:58.326483    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:39:58.348277 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:39:58.348229    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw" podStartSLOduration=1.8194511530000002 podStartE2EDuration="12.348213367s" totalImagesPullingTime="10.528762214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:39:58.346440832 +0000 UTC m=+349.074257840" watchObservedRunningTime="2026-09-08 14:39:58.348213367 +0000 UTC m=+349.076030380"
Sep 08 14:40:06.012546 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:40:06.012466    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:06.079480 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:40:06.079445    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:09.333326 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:40:09.333292    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84fb6884f5-xddmw"
Sep 08 14:40:40.823283 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:40:40.823249    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:40.975364 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:40:40.975332    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:55.309780 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:40:55.309744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:57.693448 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:40:57.693411    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:58.092418 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:40:58.092382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:58.828768 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:40:58.828736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:58.931768 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:40:58.931732    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:40:59.537651 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:40:59.537615    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:00.700310 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:00.700278    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:00.959292 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:00.959200    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:02.526220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:02.526187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:03.127759 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:03.127727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:03.229197 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:03.229156    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:03.525567 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:03.525538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:03.612431 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:03.612399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:03.939168 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:03.939078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:04.102036 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:04.101996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:09.115398 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:09.115361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:09.213037 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:09.212999    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:09.453865 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:09.453772    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cb87dcf7c-tn4d9"]
Sep 08 14:41:09.587008 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:09.586974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:09.691127 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:09.691090    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:09.787351 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:09.787316    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:16.282501 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:16.282470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:16.382796 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:16.382766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:16.746869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:16.746817    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:16.854850 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:16.854798    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:21.474216 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:21.474182    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:21.574201 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:21.574167    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:21.674655 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:21.674621    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:28.193430 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:28.193396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:28.284865 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:28.284814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:33.574400 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:33.574321    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:33.817218 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:33.817182    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:33.964321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:33.964245    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:34.052270 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.052237    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:34.475756 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.475713    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cb87dcf7c-tn4d9" podUID="5094fd28-99da-45c7-b98d-3634c9e838ee" containerName="console" containerID="cri-o://76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a" gracePeriod=15
Sep 08 14:41:34.717181 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.717160    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:41:34.752384 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.752302    2583 generic.go:356] "Generic (PLEG): container finished" podID="5094fd28-99da-45c7-b98d-3634c9e838ee" containerID="76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a" exitCode=0
Sep 08 14:41:34.752384 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.752343    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb87dcf7c-tn4d9"
Sep 08 14:41:34.752538 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.752379    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb87dcf7c-tn4d9" event={"ID":"5094fd28-99da-45c7-b98d-3634c9e838ee","Type":"ContainerDied","Data":"76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a"}
Sep 08 14:41:34.752538 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.752435    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb87dcf7c-tn4d9" event={"ID":"5094fd28-99da-45c7-b98d-3634c9e838ee","Type":"ContainerDied","Data":"698ac568b4dc469df3e77e6adc7422723586babf279f7020a524d367ff4b166d"}
Sep 08 14:41:34.752538 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.752461    2583 scope.go:117] "RemoveContainer" containerID="76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a"
Sep 08 14:41:34.760802 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.760785    2583 scope.go:117] "RemoveContainer" containerID="76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a"
Sep 08 14:41:34.761094 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:41:34.761077    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a\": container with ID starting with 76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a not found: ID does not exist" containerID="76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a"
Sep 08 14:41:34.761157 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.761101    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a"} err="failed to get container status \"76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a\": rpc error: code = NotFound desc = could not find container \"76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a\": container with ID starting with 76a38a4818907061ff9600f032e2ca78182d3a6d100f59c74a71df148c61a45a not found: ID does not exist"
Sep 08 14:41:34.784465 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.784443    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-oauth-serving-cert\") pod \"5094fd28-99da-45c7-b98d-3634c9e838ee\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") "
Sep 08 14:41:34.784549 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.784487    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-service-ca\") pod \"5094fd28-99da-45c7-b98d-3634c9e838ee\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") "
Sep 08 14:41:34.784549 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.784513    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-trusted-ca-bundle\") pod \"5094fd28-99da-45c7-b98d-3634c9e838ee\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") "
Sep 08 14:41:34.784549 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.784541    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-serving-cert\") pod \"5094fd28-99da-45c7-b98d-3634c9e838ee\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") "
Sep 08 14:41:34.784697 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.784570    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-oauth-config\") pod \"5094fd28-99da-45c7-b98d-3634c9e838ee\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") "
Sep 08 14:41:34.784697 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.784593    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-console-config\") pod \"5094fd28-99da-45c7-b98d-3634c9e838ee\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") "
Sep 08 14:41:34.784697 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.784619    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5ww8\" (UniqueName: \"kubernetes.io/projected/5094fd28-99da-45c7-b98d-3634c9e838ee-kube-api-access-d5ww8\") pod \"5094fd28-99da-45c7-b98d-3634c9e838ee\" (UID: \"5094fd28-99da-45c7-b98d-3634c9e838ee\") "
Sep 08 14:41:34.784986 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.784967    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5094fd28-99da-45c7-b98d-3634c9e838ee" (UID: "5094fd28-99da-45c7-b98d-3634c9e838ee"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:41:34.785075 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.784997    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5094fd28-99da-45c7-b98d-3634c9e838ee" (UID: "5094fd28-99da-45c7-b98d-3634c9e838ee"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:41:34.785075 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.785051    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-service-ca" (OuterVolumeSpecName: "service-ca") pod "5094fd28-99da-45c7-b98d-3634c9e838ee" (UID: "5094fd28-99da-45c7-b98d-3634c9e838ee"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:41:34.785149 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.785081    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-console-config" (OuterVolumeSpecName: "console-config") pod "5094fd28-99da-45c7-b98d-3634c9e838ee" (UID: "5094fd28-99da-45c7-b98d-3634c9e838ee"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:41:34.786873 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.786825    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5094fd28-99da-45c7-b98d-3634c9e838ee" (UID: "5094fd28-99da-45c7-b98d-3634c9e838ee"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:41:34.787042 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.787013    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5094fd28-99da-45c7-b98d-3634c9e838ee" (UID: "5094fd28-99da-45c7-b98d-3634c9e838ee"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:41:34.787113 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.787086    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5094fd28-99da-45c7-b98d-3634c9e838ee-kube-api-access-d5ww8" (OuterVolumeSpecName: "kube-api-access-d5ww8") pod "5094fd28-99da-45c7-b98d-3634c9e838ee" (UID: "5094fd28-99da-45c7-b98d-3634c9e838ee"). InnerVolumeSpecName "kube-api-access-d5ww8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:41:34.885761 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.885725    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:41:34.885761 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.885758    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-trusted-ca-bundle\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:41:34.885949 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.885773    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-serving-cert\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:41:34.885949 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.885785    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5094fd28-99da-45c7-b98d-3634c9e838ee-console-oauth-config\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:41:34.885949 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.885797    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-console-config\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:41:34.885949 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.885809    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5ww8\" (UniqueName: \"kubernetes.io/projected/5094fd28-99da-45c7-b98d-3634c9e838ee-kube-api-access-d5ww8\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:41:34.885949 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:34.885823    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5094fd28-99da-45c7-b98d-3634c9e838ee-oauth-serving-cert\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:41:35.073884 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:35.073854    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cb87dcf7c-tn4d9"]
Sep 08 14:41:35.077935 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:35.077911    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cb87dcf7c-tn4d9"]
Sep 08 14:41:35.792952 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:35.792918    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5094fd28-99da-45c7-b98d-3634c9e838ee" path="/var/lib/kubelet/pods/5094fd28-99da-45c7-b98d-3634c9e838ee/volumes"
Sep 08 14:41:37.363027 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:37.362989    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:37.467153 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:37.467109    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:41.330275 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:41.330240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:41.419838 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:41.419808    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:41.521948 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:41.521918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:53.921008 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:53.920968    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:41:54.076024 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:41:54.075996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:11.044991 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.044956    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:11.148476 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.148434    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:11.206390 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.206360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:11.883261 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.883227    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-59b9f"]
Sep 08 14:42:11.883593 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.883580    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5094fd28-99da-45c7-b98d-3634c9e838ee" containerName="console"
Sep 08 14:42:11.883651 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.883595    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5094fd28-99da-45c7-b98d-3634c9e838ee" containerName="console"
Sep 08 14:42:11.883689 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.883663    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5094fd28-99da-45c7-b98d-3634c9e838ee" containerName="console"
Sep 08 14:42:11.887241 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.887221    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-59b9f"
Sep 08 14:42:11.890095 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.890078    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 14:42:11.890182 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.890110    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 14:42:11.891253 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.891235    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-x9k59\""
Sep 08 14:42:11.896495 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.896474    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-59b9f"]
Sep 08 14:42:11.921064 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.921025    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:11.984551 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.984460    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7ftx\" (UniqueName: \"kubernetes.io/projected/a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd-kube-api-access-f7ftx\") pod \"authorino-b54df94b4-59b9f\" (UID: \"a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd\") " pod="kuadrant-system/authorino-b54df94b4-59b9f"
Sep 08 14:42:11.995069 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.995037    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:11.995475 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.995457    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-6ptbv"]
Sep 08 14:42:11.998697 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:11.998679    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-6ptbv"
Sep 08 14:42:12.006734 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.006713    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-6ptbv"]
Sep 08 14:42:12.085647 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.085622    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x42gr\" (UniqueName: \"kubernetes.io/projected/8c0f9025-4695-486e-b592-1f604fd9a69d-kube-api-access-x42gr\") pod \"authorino-c548b5d7d-6ptbv\" (UID: \"8c0f9025-4695-486e-b592-1f604fd9a69d\") " pod="kuadrant-system/authorino-c548b5d7d-6ptbv"
Sep 08 14:42:12.085974 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.085671    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7ftx\" (UniqueName: \"kubernetes.io/projected/a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd-kube-api-access-f7ftx\") pod \"authorino-b54df94b4-59b9f\" (UID: \"a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd\") " pod="kuadrant-system/authorino-b54df94b4-59b9f"
Sep 08 14:42:12.094020 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.094004    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7ftx\" (UniqueName: \"kubernetes.io/projected/a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd-kube-api-access-f7ftx\") pod \"authorino-b54df94b4-59b9f\" (UID: \"a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd\") " pod="kuadrant-system/authorino-b54df94b4-59b9f"
Sep 08 14:42:12.186413 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.186351    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x42gr\" (UniqueName: \"kubernetes.io/projected/8c0f9025-4695-486e-b592-1f604fd9a69d-kube-api-access-x42gr\") pod \"authorino-c548b5d7d-6ptbv\" (UID: \"8c0f9025-4695-486e-b592-1f604fd9a69d\") " pod="kuadrant-system/authorino-c548b5d7d-6ptbv"
Sep 08 14:42:12.195460 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.195439    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x42gr\" (UniqueName: \"kubernetes.io/projected/8c0f9025-4695-486e-b592-1f604fd9a69d-kube-api-access-x42gr\") pod \"authorino-c548b5d7d-6ptbv\" (UID: \"8c0f9025-4695-486e-b592-1f604fd9a69d\") " pod="kuadrant-system/authorino-c548b5d7d-6ptbv"
Sep 08 14:42:12.197197 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.197183    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-59b9f"
Sep 08 14:42:12.308395 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.308362    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-6ptbv"
Sep 08 14:42:12.467066 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.466997    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-59b9f"]
Sep 08 14:42:12.470353 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:42:12.470330    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda560fe19_b6c5_48d0_bf34_8d4bd0fd26cd.slice/crio-a082be8abfad535ff7ef382df56e68e08aff6bdfed0f8f780830747c248d7600 WatchSource:0}: Error finding container a082be8abfad535ff7ef382df56e68e08aff6bdfed0f8f780830747c248d7600: Status 404 returned error can't find the container with id a082be8abfad535ff7ef382df56e68e08aff6bdfed0f8f780830747c248d7600
Sep 08 14:42:12.578608 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.578587    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-6ptbv"]
Sep 08 14:42:12.580543 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:42:12.580517    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c0f9025_4695_486e_b592_1f604fd9a69d.slice/crio-a64da5c513af25c3c2847ed27d273c6d65bf11a4ba94e6089b28758cc7c868e0 WatchSource:0}: Error finding container a64da5c513af25c3c2847ed27d273c6d65bf11a4ba94e6089b28758cc7c868e0: Status 404 returned error can't find the container with id a64da5c513af25c3c2847ed27d273c6d65bf11a4ba94e6089b28758cc7c868e0
Sep 08 14:42:12.922471 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.922429    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-6ptbv" event={"ID":"8c0f9025-4695-486e-b592-1f604fd9a69d","Type":"ContainerStarted","Data":"a64da5c513af25c3c2847ed27d273c6d65bf11a4ba94e6089b28758cc7c868e0"}
Sep 08 14:42:12.923533 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:12.923508    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-59b9f" event={"ID":"a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd","Type":"ContainerStarted","Data":"a082be8abfad535ff7ef382df56e68e08aff6bdfed0f8f780830747c248d7600"}
Sep 08 14:42:16.946535 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:16.946502    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-6ptbv" event={"ID":"8c0f9025-4695-486e-b592-1f604fd9a69d","Type":"ContainerStarted","Data":"0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558"}
Sep 08 14:42:16.948021 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:16.947997    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-59b9f" event={"ID":"a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd","Type":"ContainerStarted","Data":"e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64"}
Sep 08 14:42:16.962691 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:16.962650    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-6ptbv" podStartSLOduration=2.636270089 podStartE2EDuration="5.962636596s" totalImagesPullingTime="3.326366507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:42:16.962031591 +0000 UTC m=+487.689848606" watchObservedRunningTime="2026-09-08 14:42:16.962636596 +0000 UTC m=+487.690453611"
Sep 08 14:42:16.978404 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:16.978349    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-59b9f" podStartSLOduration=2.531570639 podStartE2EDuration="5.978329531s" totalImagesPullingTime="3.446758892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:42:16.976980182 +0000 UTC m=+487.704797197" watchObservedRunningTime="2026-09-08 14:42:16.978329531 +0000 UTC m=+487.706146545"
Sep 08 14:42:16.988758 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:16.988726    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-59b9f"]
Sep 08 14:42:18.957215 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:18.957169    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-59b9f" podUID="a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd" containerName="authorino" containerID="cri-o://e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64" gracePeriod=30
Sep 08 14:42:19.197252 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.197230    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-59b9f"
Sep 08 14:42:19.347781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.347755    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7ftx\" (UniqueName: \"kubernetes.io/projected/a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd-kube-api-access-f7ftx\") pod \"a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd\" (UID: \"a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd\") "
Sep 08 14:42:19.350104 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.350071    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd-kube-api-access-f7ftx" (OuterVolumeSpecName: "kube-api-access-f7ftx") pod "a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd" (UID: "a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd"). InnerVolumeSpecName "kube-api-access-f7ftx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:42:19.448639 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.448610    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7ftx\" (UniqueName: \"kubernetes.io/projected/a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd-kube-api-access-f7ftx\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:42:19.962858 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.962816    2583 generic.go:356] "Generic (PLEG): container finished" podID="a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd" containerID="e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64" exitCode=0
Sep 08 14:42:19.963258 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.962857    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-59b9f" event={"ID":"a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd","Type":"ContainerDied","Data":"e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64"}
Sep 08 14:42:19.963258 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.962879    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-59b9f"
Sep 08 14:42:19.963258 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.962897    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-59b9f" event={"ID":"a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd","Type":"ContainerDied","Data":"a082be8abfad535ff7ef382df56e68e08aff6bdfed0f8f780830747c248d7600"}
Sep 08 14:42:19.963258 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.962918    2583 scope.go:117] "RemoveContainer" containerID="e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64"
Sep 08 14:42:19.971354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.971335    2583 scope.go:117] "RemoveContainer" containerID="e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64"
Sep 08 14:42:19.971646 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:42:19.971626    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64\": container with ID starting with e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64 not found: ID does not exist" containerID="e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64"
Sep 08 14:42:19.971707 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.971654    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64"} err="failed to get container status \"e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64\": rpc error: code = NotFound desc = could not find container \"e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64\": container with ID starting with e860bdf897d8eb9f87734b9773e0c6d7ea8316c9a5e58dceade86a5030924f64 not found: ID does not exist"
Sep 08 14:42:19.982465 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.982440    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-59b9f"]
Sep 08 14:42:19.984173 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:19.984154    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-59b9f"]
Sep 08 14:42:21.792391 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:21.792347    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd" path="/var/lib/kubelet/pods/a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd/volumes"
Sep 08 14:42:28.013772 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:28.013740    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:28.114738 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:28.114706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:28.497071 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:28.497039    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:30.146106 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:30.146072    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:41.042887 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.042826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:41.264977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.264947    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxctz"]
Sep 08 14:42:41.265379 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.265361    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd" containerName="authorino"
Sep 08 14:42:41.265468 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.265382    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd" containerName="authorino"
Sep 08 14:42:41.265520 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.265487    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a560fe19-b6c5-48d0-bf34-8d4bd0fd26cd" containerName="authorino"
Sep 08 14:42:41.268529 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.268508    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-fxctz"
Sep 08 14:42:41.275645 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.275389    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxctz"]
Sep 08 14:42:41.321938 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.321858    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v95b\" (UniqueName: \"kubernetes.io/projected/b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8-kube-api-access-9v95b\") pod \"authorino-5dc7ccb5f-fxctz\" (UID: \"b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8\") " pod="kuadrant-system/authorino-5dc7ccb5f-fxctz"
Sep 08 14:42:41.423225 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.423191    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v95b\" (UniqueName: \"kubernetes.io/projected/b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8-kube-api-access-9v95b\") pod \"authorino-5dc7ccb5f-fxctz\" (UID: \"b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8\") " pod="kuadrant-system/authorino-5dc7ccb5f-fxctz"
Sep 08 14:42:41.431491 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.431463    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v95b\" (UniqueName: \"kubernetes.io/projected/b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8-kube-api-access-9v95b\") pod \"authorino-5dc7ccb5f-fxctz\" (UID: \"b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8\") " pod="kuadrant-system/authorino-5dc7ccb5f-fxctz"
Sep 08 14:42:41.498252 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.498220    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxctz"]
Sep 08 14:42:41.498459 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.498447    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-fxctz"
Sep 08 14:42:41.525139 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.525111    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d68fb98c5-s2c7z"]
Sep 08 14:42:41.528539 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.528517    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d68fb98c5-s2c7z"
Sep 08 14:42:41.538516 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.538489    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d68fb98c5-s2c7z"]
Sep 08 14:42:41.625810 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.625729    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbkwc\" (UniqueName: \"kubernetes.io/projected/17460bc5-90ba-4680-a651-93c67ef2937b-kube-api-access-kbkwc\") pod \"authorino-5d68fb98c5-s2c7z\" (UID: \"17460bc5-90ba-4680-a651-93c67ef2937b\") " pod="kuadrant-system/authorino-5d68fb98c5-s2c7z"
Sep 08 14:42:41.656606 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.656572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:41.726961 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.726923    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbkwc\" (UniqueName: \"kubernetes.io/projected/17460bc5-90ba-4680-a651-93c67ef2937b-kube-api-access-kbkwc\") pod \"authorino-5d68fb98c5-s2c7z\" (UID: \"17460bc5-90ba-4680-a651-93c67ef2937b\") " pod="kuadrant-system/authorino-5d68fb98c5-s2c7z"
Sep 08 14:42:41.736203 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.736180    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbkwc\" (UniqueName: \"kubernetes.io/projected/17460bc5-90ba-4680-a651-93c67ef2937b-kube-api-access-kbkwc\") pod \"authorino-5d68fb98c5-s2c7z\" (UID: \"17460bc5-90ba-4680-a651-93c67ef2937b\") " pod="kuadrant-system/authorino-5d68fb98c5-s2c7z"
Sep 08 14:42:41.770382 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.770359    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxctz"]
Sep 08 14:42:41.772916 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:42:41.772881    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2b3dedb_31a6_4ed8_8a30_f2320c0d47f8.slice/crio-10ba4ae4f9da6077be1b47363e6cd972a67754ccc995b6fdb80074dede4db5ec WatchSource:0}: Error finding container 10ba4ae4f9da6077be1b47363e6cd972a67754ccc995b6fdb80074dede4db5ec: Status 404 returned error can't find the container with id 10ba4ae4f9da6077be1b47363e6cd972a67754ccc995b6fdb80074dede4db5ec
Sep 08 14:42:41.839106 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:41.839076    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d68fb98c5-s2c7z"
Sep 08 14:42:42.060637 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:42.060606    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-fxctz" event={"ID":"b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8","Type":"ContainerStarted","Data":"10ba4ae4f9da6077be1b47363e6cd972a67754ccc995b6fdb80074dede4db5ec"}
Sep 08 14:42:42.108592 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:42.108566    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d68fb98c5-s2c7z"]
Sep 08 14:42:42.111046 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:42:42.111023    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17460bc5_90ba_4680_a651_93c67ef2937b.slice/crio-1b3650246da82b0cd2e517005e69fdee10102302d1abcd54ecbe8e594b033952 WatchSource:0}: Error finding container 1b3650246da82b0cd2e517005e69fdee10102302d1abcd54ecbe8e594b033952: Status 404 returned error can't find the container with id 1b3650246da82b0cd2e517005e69fdee10102302d1abcd54ecbe8e594b033952
Sep 08 14:42:43.066781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.066751    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d68fb98c5-s2c7z" event={"ID":"17460bc5-90ba-4680-a651-93c67ef2937b","Type":"ContainerStarted","Data":"c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149"}
Sep 08 14:42:43.067215 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.066791    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d68fb98c5-s2c7z" event={"ID":"17460bc5-90ba-4680-a651-93c67ef2937b","Type":"ContainerStarted","Data":"1b3650246da82b0cd2e517005e69fdee10102302d1abcd54ecbe8e594b033952"}
Sep 08 14:42:43.068367 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.068343    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-fxctz" event={"ID":"b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8","Type":"ContainerStarted","Data":"182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7"}
Sep 08 14:42:43.068452 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.068407    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5dc7ccb5f-fxctz" podUID="b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8" containerName="authorino" containerID="cri-o://182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7" gracePeriod=30
Sep 08 14:42:43.083239 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.083198    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d68fb98c5-s2c7z" podStartSLOduration=1.588921315 podStartE2EDuration="2.083183931s" totalImagesPullingTime="494.262616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:42:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:42:43.082627449 +0000 UTC m=+513.810444464" watchObservedRunningTime="2026-09-08 14:42:43.083183931 +0000 UTC m=+513.811000945"
Sep 08 14:42:43.099035 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.098983    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5dc7ccb5f-fxctz" podStartSLOduration=1.667868342 podStartE2EDuration="2.098966932s" totalImagesPullingTime="431.09859ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:42:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:42:43.097399439 +0000 UTC m=+513.825216453" watchObservedRunningTime="2026-09-08 14:42:43.098966932 +0000 UTC m=+513.826783950"
Sep 08 14:42:43.111967 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.111939    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-6ptbv"]
Sep 08 14:42:43.112157 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.112133    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-6ptbv" podUID="8c0f9025-4695-486e-b592-1f604fd9a69d" containerName="authorino" containerID="cri-o://0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558" gracePeriod=30
Sep 08 14:42:43.185339 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.185306    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66bf74965-m4l4t"]
Sep 08 14:42:43.188428 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.188412    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66bf74965-m4l4t"
Sep 08 14:42:43.190993 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.190961    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 08 14:42:43.196807 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.196780    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66bf74965-m4l4t"]
Sep 08 14:42:43.241044 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.241013    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/226a52ae-b256-43ac-9424-b9e5c7ca2b93-tls-cert\") pod \"authorino-66bf74965-m4l4t\" (UID: \"226a52ae-b256-43ac-9424-b9e5c7ca2b93\") " pod="kuadrant-system/authorino-66bf74965-m4l4t"
Sep 08 14:42:43.241170 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.241056    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdw6k\" (UniqueName: \"kubernetes.io/projected/226a52ae-b256-43ac-9424-b9e5c7ca2b93-kube-api-access-kdw6k\") pod \"authorino-66bf74965-m4l4t\" (UID: \"226a52ae-b256-43ac-9424-b9e5c7ca2b93\") " pod="kuadrant-system/authorino-66bf74965-m4l4t"
Sep 08 14:42:43.341642 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.341575    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/226a52ae-b256-43ac-9424-b9e5c7ca2b93-tls-cert\") pod \"authorino-66bf74965-m4l4t\" (UID: \"226a52ae-b256-43ac-9424-b9e5c7ca2b93\") " pod="kuadrant-system/authorino-66bf74965-m4l4t"
Sep 08 14:42:43.341642 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.341618    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdw6k\" (UniqueName: \"kubernetes.io/projected/226a52ae-b256-43ac-9424-b9e5c7ca2b93-kube-api-access-kdw6k\") pod \"authorino-66bf74965-m4l4t\" (UID: \"226a52ae-b256-43ac-9424-b9e5c7ca2b93\") " pod="kuadrant-system/authorino-66bf74965-m4l4t"
Sep 08 14:42:43.344103 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.344078    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/226a52ae-b256-43ac-9424-b9e5c7ca2b93-tls-cert\") pod \"authorino-66bf74965-m4l4t\" (UID: \"226a52ae-b256-43ac-9424-b9e5c7ca2b93\") " pod="kuadrant-system/authorino-66bf74965-m4l4t"
Sep 08 14:42:43.351180 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.351161    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdw6k\" (UniqueName: \"kubernetes.io/projected/226a52ae-b256-43ac-9424-b9e5c7ca2b93-kube-api-access-kdw6k\") pod \"authorino-66bf74965-m4l4t\" (UID: \"226a52ae-b256-43ac-9424-b9e5c7ca2b93\") " pod="kuadrant-system/authorino-66bf74965-m4l4t"
Sep 08 14:42:43.363379 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.363361    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-fxctz"
Sep 08 14:42:43.369013 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.368995    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-6ptbv"
Sep 08 14:42:43.442358 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.442335    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x42gr\" (UniqueName: \"kubernetes.io/projected/8c0f9025-4695-486e-b592-1f604fd9a69d-kube-api-access-x42gr\") pod \"8c0f9025-4695-486e-b592-1f604fd9a69d\" (UID: \"8c0f9025-4695-486e-b592-1f604fd9a69d\") "
Sep 08 14:42:43.442450 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.442425    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v95b\" (UniqueName: \"kubernetes.io/projected/b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8-kube-api-access-9v95b\") pod \"b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8\" (UID: \"b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8\") "
Sep 08 14:42:43.444549 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.444529    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c0f9025-4695-486e-b592-1f604fd9a69d-kube-api-access-x42gr" (OuterVolumeSpecName: "kube-api-access-x42gr") pod "8c0f9025-4695-486e-b592-1f604fd9a69d" (UID: "8c0f9025-4695-486e-b592-1f604fd9a69d"). InnerVolumeSpecName "kube-api-access-x42gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:42:43.444713 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.444688    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8-kube-api-access-9v95b" (OuterVolumeSpecName: "kube-api-access-9v95b") pod "b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8" (UID: "b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8"). InnerVolumeSpecName "kube-api-access-9v95b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:42:43.497881 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.497825    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66bf74965-m4l4t"
Sep 08 14:42:43.543187 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.543154    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x42gr\" (UniqueName: \"kubernetes.io/projected/8c0f9025-4695-486e-b592-1f604fd9a69d-kube-api-access-x42gr\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:42:43.543187 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.543189    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v95b\" (UniqueName: \"kubernetes.io/projected/b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8-kube-api-access-9v95b\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:42:43.766528 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:43.766495    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66bf74965-m4l4t"]
Sep 08 14:42:43.769784 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:42:43.769759    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod226a52ae_b256_43ac_9424_b9e5c7ca2b93.slice/crio-1d80dc97865c451eca458db648a31ddd028b37fa2f4d749afb69fcbccbf93c60 WatchSource:0}: Error finding container 1d80dc97865c451eca458db648a31ddd028b37fa2f4d749afb69fcbccbf93c60: Status 404 returned error can't find the container with id 1d80dc97865c451eca458db648a31ddd028b37fa2f4d749afb69fcbccbf93c60
Sep 08 14:42:44.074505 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.074398    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66bf74965-m4l4t" event={"ID":"226a52ae-b256-43ac-9424-b9e5c7ca2b93","Type":"ContainerStarted","Data":"1d80dc97865c451eca458db648a31ddd028b37fa2f4d749afb69fcbccbf93c60"}
Sep 08 14:42:44.075693 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.075670    2583 generic.go:356] "Generic (PLEG): container finished" podID="8c0f9025-4695-486e-b592-1f604fd9a69d" containerID="0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558" exitCode=0
Sep 08 14:42:44.075755 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.075725    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-6ptbv"
Sep 08 14:42:44.075806 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.075752    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-6ptbv" event={"ID":"8c0f9025-4695-486e-b592-1f604fd9a69d","Type":"ContainerDied","Data":"0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558"}
Sep 08 14:42:44.075806 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.075799    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-6ptbv" event={"ID":"8c0f9025-4695-486e-b592-1f604fd9a69d","Type":"ContainerDied","Data":"a64da5c513af25c3c2847ed27d273c6d65bf11a4ba94e6089b28758cc7c868e0"}
Sep 08 14:42:44.075917 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.075824    2583 scope.go:117] "RemoveContainer" containerID="0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558"
Sep 08 14:42:44.077099 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.077072    2583 generic.go:356] "Generic (PLEG): container finished" podID="b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8" containerID="182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7" exitCode=0
Sep 08 14:42:44.077198 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.077106    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-fxctz" event={"ID":"b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8","Type":"ContainerDied","Data":"182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7"}
Sep 08 14:42:44.077198 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.077131    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-fxctz"
Sep 08 14:42:44.077198 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.077147    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-fxctz" event={"ID":"b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8","Type":"ContainerDied","Data":"10ba4ae4f9da6077be1b47363e6cd972a67754ccc995b6fdb80074dede4db5ec"}
Sep 08 14:42:44.084259 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.084243    2583 scope.go:117] "RemoveContainer" containerID="0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558"
Sep 08 14:42:44.084508 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:42:44.084490    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558\": container with ID starting with 0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558 not found: ID does not exist" containerID="0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558"
Sep 08 14:42:44.084557 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.084516    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558"} err="failed to get container status \"0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558\": rpc error: code = NotFound desc = could not find container \"0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558\": container with ID starting with 0b5b1cc1ef570888bf8ec683f786258028bbe96254d62069fadd99d22f8a2558 not found: ID does not exist"
Sep 08 14:42:44.084557 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.084532    2583 scope.go:117] "RemoveContainer" containerID="182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7"
Sep 08 14:42:44.092503 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.092484    2583 scope.go:117] "RemoveContainer" containerID="182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7"
Sep 08 14:42:44.092785 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:42:44.092762    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7\": container with ID starting with 182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7 not found: ID does not exist" containerID="182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7"
Sep 08 14:42:44.092919 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.092801    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7"} err="failed to get container status \"182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7\": rpc error: code = NotFound desc = could not find container \"182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7\": container with ID starting with 182ae28a4cb55b0dd5d5ce910d557fe4ba51e87013861560ae87a1d45c426cb7 not found: ID does not exist"
Sep 08 14:42:44.093638 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.093616    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-6ptbv"]
Sep 08 14:42:44.097215 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.097194    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-6ptbv"]
Sep 08 14:42:44.109880 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.109860    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxctz"]
Sep 08 14:42:44.113451 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:44.113431    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxctz"]
Sep 08 14:42:45.087562 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:45.087519    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66bf74965-m4l4t" event={"ID":"226a52ae-b256-43ac-9424-b9e5c7ca2b93","Type":"ContainerStarted","Data":"bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b"}
Sep 08 14:42:45.103685 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:45.103635    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66bf74965-m4l4t" podStartSLOduration=1.49748567 podStartE2EDuration="2.103619429s" totalImagesPullingTime="606.133759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:42:45.102810461 +0000 UTC m=+515.830627499" watchObservedRunningTime="2026-09-08 14:42:45.103619429 +0000 UTC m=+515.831436442"
Sep 08 14:42:45.130543 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:45.130506    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d68fb98c5-s2c7z"]
Sep 08 14:42:45.130691 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:45.130673    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d68fb98c5-s2c7z" podUID="17460bc5-90ba-4680-a651-93c67ef2937b" containerName="authorino" containerID="cri-o://c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149" gracePeriod=30
Sep 08 14:42:45.368325 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:45.368302    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d68fb98c5-s2c7z"
Sep 08 14:42:45.460007 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:45.459975    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbkwc\" (UniqueName: \"kubernetes.io/projected/17460bc5-90ba-4680-a651-93c67ef2937b-kube-api-access-kbkwc\") pod \"17460bc5-90ba-4680-a651-93c67ef2937b\" (UID: \"17460bc5-90ba-4680-a651-93c67ef2937b\") "
Sep 08 14:42:45.462133 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:45.462101    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17460bc5-90ba-4680-a651-93c67ef2937b-kube-api-access-kbkwc" (OuterVolumeSpecName: "kube-api-access-kbkwc") pod "17460bc5-90ba-4680-a651-93c67ef2937b" (UID: "17460bc5-90ba-4680-a651-93c67ef2937b"). InnerVolumeSpecName "kube-api-access-kbkwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:42:45.561191 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:45.561161    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbkwc\" (UniqueName: \"kubernetes.io/projected/17460bc5-90ba-4680-a651-93c67ef2937b-kube-api-access-kbkwc\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:42:45.792641 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:45.792609    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c0f9025-4695-486e-b592-1f604fd9a69d" path="/var/lib/kubelet/pods/8c0f9025-4695-486e-b592-1f604fd9a69d/volumes"
Sep 08 14:42:45.792960 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:45.792946    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8" path="/var/lib/kubelet/pods/b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8/volumes"
Sep 08 14:42:46.092895 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:46.092808    2583 generic.go:356] "Generic (PLEG): container finished" podID="17460bc5-90ba-4680-a651-93c67ef2937b" containerID="c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149" exitCode=0
Sep 08 14:42:46.092895 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:46.092877    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d68fb98c5-s2c7z"
Sep 08 14:42:46.093442 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:46.092901    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d68fb98c5-s2c7z" event={"ID":"17460bc5-90ba-4680-a651-93c67ef2937b","Type":"ContainerDied","Data":"c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149"}
Sep 08 14:42:46.093442 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:46.092938    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d68fb98c5-s2c7z" event={"ID":"17460bc5-90ba-4680-a651-93c67ef2937b","Type":"ContainerDied","Data":"1b3650246da82b0cd2e517005e69fdee10102302d1abcd54ecbe8e594b033952"}
Sep 08 14:42:46.093442 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:46.092953    2583 scope.go:117] "RemoveContainer" containerID="c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149"
Sep 08 14:42:46.101251 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:46.101236    2583 scope.go:117] "RemoveContainer" containerID="c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149"
Sep 08 14:42:46.101531 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:42:46.101512    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149\": container with ID starting with c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149 not found: ID does not exist" containerID="c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149"
Sep 08 14:42:46.101585 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:46.101539    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149"} err="failed to get container status \"c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149\": rpc error: code = NotFound desc = could not find container \"c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149\": container with ID starting with c5ec7f30dbfedc5dbae7000eae07484b64b696cf7966c8f2d74dd7d2160da149 not found: ID does not exist"
Sep 08 14:42:46.109378 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:46.109353    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d68fb98c5-s2c7z"]
Sep 08 14:42:46.115446 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:46.115424    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d68fb98c5-s2c7z"]
Sep 08 14:42:47.793239 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:47.793211    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17460bc5-90ba-4680-a651-93c67ef2937b" path="/var/lib/kubelet/pods/17460bc5-90ba-4680-a651-93c67ef2937b/volumes"
Sep 08 14:42:47.901376 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:47.901349    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:47.992283 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:47.992249    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:48.049100 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:48.049024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:48.228643 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:48.228617    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:48.352028 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:48.351945    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:51.362801 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:51.362770    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:42:51.460047 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:42:51.460014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:06.053898 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:06.053795    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:06.154011 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:06.153962    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:21.856981 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:21.856943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:23.892171 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:23.892133    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:23.987811 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:23.987776    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:25.383395 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:25.383362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:25.485005 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:25.484968    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:25.921352 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:25.921315    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:26.016226 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.016195    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:26.047582 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.047539    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"]
Sep 08 14:43:26.047944 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.047923    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8" containerName="authorino"
Sep 08 14:43:26.047944 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.047944    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8" containerName="authorino"
Sep 08 14:43:26.048101 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.047974    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c0f9025-4695-486e-b592-1f604fd9a69d" containerName="authorino"
Sep 08 14:43:26.048101 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.047980    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c0f9025-4695-486e-b592-1f604fd9a69d" containerName="authorino"
Sep 08 14:43:26.048101 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.047988    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17460bc5-90ba-4680-a651-93c67ef2937b" containerName="authorino"
Sep 08 14:43:26.048101 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.047993    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="17460bc5-90ba-4680-a651-93c67ef2937b" containerName="authorino"
Sep 08 14:43:26.048101 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.048075    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c0f9025-4695-486e-b592-1f604fd9a69d" containerName="authorino"
Sep 08 14:43:26.048101 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.048087    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2b3dedb-31a6-4ed8-8a30-f2320c0d47f8" containerName="authorino"
Sep 08 14:43:26.048101 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.048095    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="17460bc5-90ba-4680-a651-93c67ef2937b" containerName="authorino"
Sep 08 14:43:26.052733 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.052711    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"
Sep 08 14:43:26.056304 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.056283    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 08 14:43:26.064516 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.064492    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"]
Sep 08 14:43:26.107469 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.107438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:26.204311 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.204237    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv7kl\" (UniqueName: \"kubernetes.io/projected/0ed5538c-88ce-4137-860a-1162e325b412-kube-api-access-gv7kl\") pod \"payload-pre-processing-59fbc47bf8-558h6\" (UID: \"0ed5538c-88ce-4137-860a-1162e325b412\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"
Sep 08 14:43:26.204445 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.204340    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ed5538c-88ce-4137-860a-1162e325b412-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-558h6\" (UID: \"0ed5538c-88ce-4137-860a-1162e325b412\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"
Sep 08 14:43:26.305406 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.305366    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ed5538c-88ce-4137-860a-1162e325b412-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-558h6\" (UID: \"0ed5538c-88ce-4137-860a-1162e325b412\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"
Sep 08 14:43:26.305590 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.305533    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv7kl\" (UniqueName: \"kubernetes.io/projected/0ed5538c-88ce-4137-860a-1162e325b412-kube-api-access-gv7kl\") pod \"payload-pre-processing-59fbc47bf8-558h6\" (UID: \"0ed5538c-88ce-4137-860a-1162e325b412\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"
Sep 08 14:43:26.306276 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.306252    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ed5538c-88ce-4137-860a-1162e325b412-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-558h6\" (UID: \"0ed5538c-88ce-4137-860a-1162e325b412\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"
Sep 08 14:43:26.315048 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.315022    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv7kl\" (UniqueName: \"kubernetes.io/projected/0ed5538c-88ce-4137-860a-1162e325b412-kube-api-access-gv7kl\") pod \"payload-pre-processing-59fbc47bf8-558h6\" (UID: \"0ed5538c-88ce-4137-860a-1162e325b412\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"
Sep 08 14:43:26.364684 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.364642    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"
Sep 08 14:43:26.653432 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:26.653406    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"]
Sep 08 14:43:26.655664 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:43:26.655639    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed5538c_88ce_4137_860a_1162e325b412.slice/crio-66c432aa2b8dc75dace3aeb929af62c96d0bcb5fa9b4e6a6f15100906879c427 WatchSource:0}: Error finding container 66c432aa2b8dc75dace3aeb929af62c96d0bcb5fa9b4e6a6f15100906879c427: Status 404 returned error can't find the container with id 66c432aa2b8dc75dace3aeb929af62c96d0bcb5fa9b4e6a6f15100906879c427
Sep 08 14:43:27.281637 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:27.281604    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6" event={"ID":"0ed5538c-88ce-4137-860a-1162e325b412","Type":"ContainerStarted","Data":"66c432aa2b8dc75dace3aeb929af62c96d0bcb5fa9b4e6a6f15100906879c427"}
Sep 08 14:43:29.292318 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:29.292282    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6" event={"ID":"0ed5538c-88ce-4137-860a-1162e325b412","Type":"ContainerStarted","Data":"b108fe134a4000d8706e3edafda247489c319aa13841534945ea0af5b621d0f8"}
Sep 08 14:43:29.292782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:29.292442    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"
Sep 08 14:43:29.309479 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:29.309434    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6" podStartSLOduration=1.6238253820000002 podStartE2EDuration="3.309420309s" totalImagesPullingTime="1.685594927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:43:29.308710411 +0000 UTC m=+560.036527425" watchObservedRunningTime="2026-09-08 14:43:29.309420309 +0000 UTC m=+560.037237320"
Sep 08 14:43:36.163285 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:36.163249    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:36.258993 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:36.258955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:43:40.298949 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:40.298922    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-558h6"
Sep 08 14:43:57.398283 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:43:57.398244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:04.558172 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.558136    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"]
Sep 08 14:44:04.560711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.560687    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.563868 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.563850    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 14:44:04.565020 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.565003    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 14:44:04.565095 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.565012    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cmtn4\""
Sep 08 14:44:04.565095 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.565012    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 08 14:44:04.579738 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.579714    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"]
Sep 08 14:44:04.596388 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.596364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:04.724634 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.724603    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.724772 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.724639    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.724772 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.724661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vltz\" (UniqueName: \"kubernetes.io/projected/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-kube-api-access-7vltz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.724772 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.724728    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.724951 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.724771    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.724951 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.724800    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.826180 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.826095    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.826180 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.826145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.826389 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.826200    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.826389 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.826233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.826389 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.826257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vltz\" (UniqueName: \"kubernetes.io/projected/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-kube-api-access-7vltz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.826389 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.826300    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.826637 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.826538    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.826637 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.826572    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.826737 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.826689    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.828552 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.828524    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.828735 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.828721    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.835111 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.835088    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vltz\" (UniqueName: \"kubernetes.io/projected/f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092-kube-api-access-7vltz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l47fw\" (UID: \"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:04.872438 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:04.872414    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:05.142874 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:05.142824    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"]
Sep 08 14:44:05.457224 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:05.457127    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw" event={"ID":"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092","Type":"ContainerStarted","Data":"50cfad4c255db0dc1aa4ae510c5b2e02f0afe5ec7b187b78b07def901ed648ac"}
Sep 08 14:44:06.463483 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:06.463440    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw" event={"ID":"f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092","Type":"ContainerStarted","Data":"8d771b7cac6dc318893e6d6460d583dd47fcfa83a9879ed1164136436936f5e1"}
Sep 08 14:44:06.484669 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:06.484622    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw" podStartSLOduration=1.384749157 podStartE2EDuration="2.484605655s" totalImagesPullingTime="1.099856498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:44:06.482630458 +0000 UTC m=+597.210447484" watchObservedRunningTime="2026-09-08 14:44:06.484605655 +0000 UTC m=+597.212422711"
Sep 08 14:44:07.719481 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:07.719444    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:11.230349 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:11.230313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:14.873189 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:14.873145    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:14.873189 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:14.873206    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:14.886283 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:14.886258    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:15.517453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:15.517424    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l47fw"
Sep 08 14:44:16.641671 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:16.641632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:20.077082 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:20.077045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:36.165873 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.165786    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"]
Sep 08 14:44:36.173294 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.173271    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.176175 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.176155    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 08 14:44:36.182596 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.182571    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"]
Sep 08 14:44:36.200046 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.200008    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:36.287716 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.287685    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkzlq\" (UniqueName: \"kubernetes.io/projected/d2365d49-879c-46cd-b363-02d80acf695c-kube-api-access-qkzlq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.287716 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.287719    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.287919 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.287824    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.287919 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.287868    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2365d49-879c-46cd-b363-02d80acf695c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.287919 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.287905    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.288033 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.287934    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.388992 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.388959    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkzlq\" (UniqueName: \"kubernetes.io/projected/d2365d49-879c-46cd-b363-02d80acf695c-kube-api-access-qkzlq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.389178 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.388999    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.389178 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.389091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.389299 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.389219    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2365d49-879c-46cd-b363-02d80acf695c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.389299 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.389264    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.389409 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.389313    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.389502 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.389483    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.389623 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.389596    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.389728 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.389641    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.391442 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.391423    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2365d49-879c-46cd-b363-02d80acf695c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.391907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.391885    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2365d49-879c-46cd-b363-02d80acf695c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.398284 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.398260    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkzlq\" (UniqueName: \"kubernetes.io/projected/d2365d49-879c-46cd-b363-02d80acf695c-kube-api-access-qkzlq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx\" (UID: \"d2365d49-879c-46cd-b363-02d80acf695c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.484257 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.484226    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:36.759845 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.759607    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"]
Sep 08 14:44:36.762197 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:44:36.762174    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2365d49_879c_46cd_b363_02d80acf695c.slice/crio-0bbd79b3510999ee4f2e6ec74b19ccbdf352da1228fcbd4f47bb7b8b006a2b65 WatchSource:0}: Error finding container 0bbd79b3510999ee4f2e6ec74b19ccbdf352da1228fcbd4f47bb7b8b006a2b65: Status 404 returned error can't find the container with id 0bbd79b3510999ee4f2e6ec74b19ccbdf352da1228fcbd4f47bb7b8b006a2b65
Sep 08 14:44:36.763487 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:36.763467    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:44:37.606016 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:37.605980    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx" event={"ID":"d2365d49-879c-46cd-b363-02d80acf695c","Type":"ContainerStarted","Data":"0aa7540f5ec257cbdc5677b127267474deb0ead457e5004f12d4817b4229f151"}
Sep 08 14:44:37.606392 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:37.606030    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx" event={"ID":"d2365d49-879c-46cd-b363-02d80acf695c","Type":"ContainerStarted","Data":"0bbd79b3510999ee4f2e6ec74b19ccbdf352da1228fcbd4f47bb7b8b006a2b65"}
Sep 08 14:44:37.624174 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:37.624126    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx" podStartSLOduration=1.408404945 podStartE2EDuration="1.624110465s" totalImagesPullingTime="215.70552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:44:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:44:37.622937255 +0000 UTC m=+628.350754268" watchObservedRunningTime="2026-09-08 14:44:37.624110465 +0000 UTC m=+628.351927479"
Sep 08 14:44:45.975304 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:45.975274    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"]
Sep 08 14:44:45.978877 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:45.978859    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:45.981956 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:45.981934    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 08 14:44:45.990248 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:45.990222    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"]
Sep 08 14:44:46.005953 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.005927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:46.073640 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.073608    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99v5g\" (UniqueName: \"kubernetes.io/projected/1ae8799f-6358-4fbb-a733-3123a72f1807-kube-api-access-99v5g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.073768 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.073645    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.073768 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.073675    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.073768 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.073759    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.073920 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.073804    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae8799f-6358-4fbb-a733-3123a72f1807-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.073920 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.073826    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.174721 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.174690    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99v5g\" (UniqueName: \"kubernetes.io/projected/1ae8799f-6358-4fbb-a733-3123a72f1807-kube-api-access-99v5g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.174721 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.174722    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.174941 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.174742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.174941 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.174803    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.174941 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.174908    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae8799f-6358-4fbb-a733-3123a72f1807-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.175091 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.175013    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.175231 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.175208    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.175296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.175229    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.175296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.175249    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.177295 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.177266    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae8799f-6358-4fbb-a733-3123a72f1807-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.177410 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.177395    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae8799f-6358-4fbb-a733-3123a72f1807-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.183712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.183695    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99v5g\" (UniqueName: \"kubernetes.io/projected/1ae8799f-6358-4fbb-a733-3123a72f1807-kube-api-access-99v5g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r4pwc\" (UID: \"1ae8799f-6358-4fbb-a733-3123a72f1807\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.289981 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.289895    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:46.484610 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.484565    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:46.484791 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.484636    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:46.497416 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.497387    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:46.562546 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.562525    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"]
Sep 08 14:44:46.565015 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:44:46.564985    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae8799f_6358_4fbb_a733_3123a72f1807.slice/crio-38251eb1fe41b3d1314ac4f3b2e7aa64714e9023a3338f26940e5ddb18075244 WatchSource:0}: Error finding container 38251eb1fe41b3d1314ac4f3b2e7aa64714e9023a3338f26940e5ddb18075244: Status 404 returned error can't find the container with id 38251eb1fe41b3d1314ac4f3b2e7aa64714e9023a3338f26940e5ddb18075244
Sep 08 14:44:46.647866 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.647814    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc" event={"ID":"1ae8799f-6358-4fbb-a733-3123a72f1807","Type":"ContainerStarted","Data":"38251eb1fe41b3d1314ac4f3b2e7aa64714e9023a3338f26940e5ddb18075244"}
Sep 08 14:44:46.658895 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:46.658875    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx"
Sep 08 14:44:47.654341 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:47.654300    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc" event={"ID":"1ae8799f-6358-4fbb-a733-3123a72f1807","Type":"ContainerStarted","Data":"01cdbe681e94428dc1b9414f8caca3e7e9a8f8cdfa52248da5d7beb0032422de"}
Sep 08 14:44:47.674561 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:47.674510    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc" podStartSLOduration=2.439568869 podStartE2EDuration="2.674494328s" totalImagesPullingTime="234.925459ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:44:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:44:47.67264897 +0000 UTC m=+638.400466009" watchObservedRunningTime="2026-09-08 14:44:47.674494328 +0000 UTC m=+638.402311341"
Sep 08 14:44:50.059706 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.059672    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"]
Sep 08 14:44:50.063189 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.063166    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.066016 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.066000    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 08 14:44:50.072909 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.072876    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"]
Sep 08 14:44:50.104372 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.104344    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:50.215041 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.215007    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.215041 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.215040    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44388934-a2dd-427d-9cb6-ca9d6d0e637d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.215232 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.215057    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9dwp\" (UniqueName: \"kubernetes.io/projected/44388934-a2dd-427d-9cb6-ca9d6d0e637d-kube-api-access-w9dwp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.215232 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.215137    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.215232 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.215177    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.215232 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.215210    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.316325 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.316243    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.316325 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.316279    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44388934-a2dd-427d-9cb6-ca9d6d0e637d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.316325 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.316302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9dwp\" (UniqueName: \"kubernetes.io/projected/44388934-a2dd-427d-9cb6-ca9d6d0e637d-kube-api-access-w9dwp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.316602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.316341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.316602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.316381    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.316602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.316528    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.316744 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.316704    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.316785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.316764    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.316927 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.316900    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.318741 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.318720    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44388934-a2dd-427d-9cb6-ca9d6d0e637d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.319060 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.319040    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44388934-a2dd-427d-9cb6-ca9d6d0e637d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.325976 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.325957    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9dwp\" (UniqueName: \"kubernetes.io/projected/44388934-a2dd-427d-9cb6-ca9d6d0e637d-kube-api-access-w9dwp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-748tl\" (UID: \"44388934-a2dd-427d-9cb6-ca9d6d0e637d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.373819 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.373792    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:44:50.646752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.646717    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"]
Sep 08 14:44:50.648975 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:44:50.648945    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44388934_a2dd_427d_9cb6_ca9d6d0e637d.slice/crio-ba6d4543eb7d205b383a54d09e8eaccac3fc959792989dc3dd39ec17b6fd51e0 WatchSource:0}: Error finding container ba6d4543eb7d205b383a54d09e8eaccac3fc959792989dc3dd39ec17b6fd51e0: Status 404 returned error can't find the container with id ba6d4543eb7d205b383a54d09e8eaccac3fc959792989dc3dd39ec17b6fd51e0
Sep 08 14:44:50.669639 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:50.669612    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl" event={"ID":"44388934-a2dd-427d-9cb6-ca9d6d0e637d","Type":"ContainerStarted","Data":"ba6d4543eb7d205b383a54d09e8eaccac3fc959792989dc3dd39ec17b6fd51e0"}
Sep 08 14:44:51.675876 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:51.675808    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl" event={"ID":"44388934-a2dd-427d-9cb6-ca9d6d0e637d","Type":"ContainerStarted","Data":"7d2500882fd951580871bb834b1254a9a43306b53a83271071aa8c4384624ef2"}
Sep 08 14:44:51.697675 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:51.697626    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl" podStartSLOduration=1.385911236 podStartE2EDuration="1.69760926s" totalImagesPullingTime="311.698024ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:44:51.69595242 +0000 UTC m=+642.423769438" watchObservedRunningTime="2026-09-08 14:44:51.69760926 +0000 UTC m=+642.425426274"
Sep 08 14:44:54.774817 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:54.774787    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:44:56.290355 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:56.290308    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:56.290745 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:56.290408    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:56.303072 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:56.303044    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:44:56.710970 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:44:56.710892    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r4pwc"
Sep 08 14:45:00.374875 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:00.374812    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:45:00.375242 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:00.374902    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:45:00.387280 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:00.387256    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:45:00.729020 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:00.728995    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-748tl"
Sep 08 14:45:32.393805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:32.393757    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:32.466347 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:32.466313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:43.179717 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.179684    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f4b698f7b-lz9v7"]
Sep 08 14:45:43.183047 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.183031    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f4b698f7b-lz9v7"
Sep 08 14:45:43.193504 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.193438    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f4b698f7b-lz9v7"]
Sep 08 14:45:43.271389 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.271360    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f47c8902-1414-4630-811b-b684dc5596fa-tls-cert\") pod \"authorino-6f4b698f7b-lz9v7\" (UID: \"f47c8902-1414-4630-811b-b684dc5596fa\") " pod="kuadrant-system/authorino-6f4b698f7b-lz9v7"
Sep 08 14:45:43.271389 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.271390    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4njdr\" (UniqueName: \"kubernetes.io/projected/f47c8902-1414-4630-811b-b684dc5596fa-kube-api-access-4njdr\") pod \"authorino-6f4b698f7b-lz9v7\" (UID: \"f47c8902-1414-4630-811b-b684dc5596fa\") " pod="kuadrant-system/authorino-6f4b698f7b-lz9v7"
Sep 08 14:45:43.372141 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.372099    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f47c8902-1414-4630-811b-b684dc5596fa-tls-cert\") pod \"authorino-6f4b698f7b-lz9v7\" (UID: \"f47c8902-1414-4630-811b-b684dc5596fa\") " pod="kuadrant-system/authorino-6f4b698f7b-lz9v7"
Sep 08 14:45:43.372141 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.372144    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4njdr\" (UniqueName: \"kubernetes.io/projected/f47c8902-1414-4630-811b-b684dc5596fa-kube-api-access-4njdr\") pod \"authorino-6f4b698f7b-lz9v7\" (UID: \"f47c8902-1414-4630-811b-b684dc5596fa\") " pod="kuadrant-system/authorino-6f4b698f7b-lz9v7"
Sep 08 14:45:43.374705 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.374685    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f47c8902-1414-4630-811b-b684dc5596fa-tls-cert\") pod \"authorino-6f4b698f7b-lz9v7\" (UID: \"f47c8902-1414-4630-811b-b684dc5596fa\") " pod="kuadrant-system/authorino-6f4b698f7b-lz9v7"
Sep 08 14:45:43.381351 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.381323    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4njdr\" (UniqueName: \"kubernetes.io/projected/f47c8902-1414-4630-811b-b684dc5596fa-kube-api-access-4njdr\") pod \"authorino-6f4b698f7b-lz9v7\" (UID: \"f47c8902-1414-4630-811b-b684dc5596fa\") " pod="kuadrant-system/authorino-6f4b698f7b-lz9v7"
Sep 08 14:45:43.492714 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.492689    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f4b698f7b-lz9v7"
Sep 08 14:45:43.763658 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.763631    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f4b698f7b-lz9v7"]
Sep 08 14:45:43.766061 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:45:43.766035    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf47c8902_1414_4630_811b_b684dc5596fa.slice/crio-ecc512dd331457acb65611264469839e00f8e45d8879d523d0af476debcf4b32 WatchSource:0}: Error finding container ecc512dd331457acb65611264469839e00f8e45d8879d523d0af476debcf4b32: Status 404 returned error can't find the container with id ecc512dd331457acb65611264469839e00f8e45d8879d523d0af476debcf4b32
Sep 08 14:45:43.920450 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:43.920418    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f4b698f7b-lz9v7" event={"ID":"f47c8902-1414-4630-811b-b684dc5596fa","Type":"ContainerStarted","Data":"ecc512dd331457acb65611264469839e00f8e45d8879d523d0af476debcf4b32"}
Sep 08 14:45:44.930304 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:44.930200    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f4b698f7b-lz9v7" event={"ID":"f47c8902-1414-4630-811b-b684dc5596fa","Type":"ContainerStarted","Data":"9f0068308aac039d00d5d53d11c42ae8f5a97fe73c49c8f93f78a77c0ebcdfc9"}
Sep 08 14:45:44.955258 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:44.955192    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6f4b698f7b-lz9v7" podStartSLOduration=1.159576202 podStartE2EDuration="1.95517004s" totalImagesPullingTime="795.593838ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:45:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:45:44.954963295 +0000 UTC m=+695.682780308" watchObservedRunningTime="2026-09-08 14:45:44.95517004 +0000 UTC m=+695.682987056"
Sep 08 14:45:44.996446 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:44.996414    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66bf74965-m4l4t"]
Sep 08 14:45:44.996660 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:44.996626    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-66bf74965-m4l4t" podUID="226a52ae-b256-43ac-9424-b9e5c7ca2b93" containerName="authorino" containerID="cri-o://bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b" gracePeriod=30
Sep 08 14:45:45.244512 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.244486    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66bf74965-m4l4t"
Sep 08 14:45:45.391520 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.391487    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdw6k\" (UniqueName: \"kubernetes.io/projected/226a52ae-b256-43ac-9424-b9e5c7ca2b93-kube-api-access-kdw6k\") pod \"226a52ae-b256-43ac-9424-b9e5c7ca2b93\" (UID: \"226a52ae-b256-43ac-9424-b9e5c7ca2b93\") "
Sep 08 14:45:45.391715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.391536    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/226a52ae-b256-43ac-9424-b9e5c7ca2b93-tls-cert\") pod \"226a52ae-b256-43ac-9424-b9e5c7ca2b93\" (UID: \"226a52ae-b256-43ac-9424-b9e5c7ca2b93\") "
Sep 08 14:45:45.393743 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.393714    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/226a52ae-b256-43ac-9424-b9e5c7ca2b93-kube-api-access-kdw6k" (OuterVolumeSpecName: "kube-api-access-kdw6k") pod "226a52ae-b256-43ac-9424-b9e5c7ca2b93" (UID: "226a52ae-b256-43ac-9424-b9e5c7ca2b93"). InnerVolumeSpecName "kube-api-access-kdw6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:45:45.402149 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.402125    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/226a52ae-b256-43ac-9424-b9e5c7ca2b93-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "226a52ae-b256-43ac-9424-b9e5c7ca2b93" (UID: "226a52ae-b256-43ac-9424-b9e5c7ca2b93"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:45:45.492929 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.492807    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdw6k\" (UniqueName: \"kubernetes.io/projected/226a52ae-b256-43ac-9424-b9e5c7ca2b93-kube-api-access-kdw6k\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:45:45.492929 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.492864    2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/226a52ae-b256-43ac-9424-b9e5c7ca2b93-tls-cert\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:45:45.574398 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.574347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:45.673300 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.673259    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:45.939571 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.939531    2583 generic.go:356] "Generic (PLEG): container finished" podID="226a52ae-b256-43ac-9424-b9e5c7ca2b93" containerID="bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b" exitCode=0
Sep 08 14:45:45.940866 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.940823    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66bf74965-m4l4t"
Sep 08 14:45:45.943719 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.943687    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66bf74965-m4l4t" event={"ID":"226a52ae-b256-43ac-9424-b9e5c7ca2b93","Type":"ContainerDied","Data":"bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b"}
Sep 08 14:45:45.943901 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.943883    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66bf74965-m4l4t" event={"ID":"226a52ae-b256-43ac-9424-b9e5c7ca2b93","Type":"ContainerDied","Data":"1d80dc97865c451eca458db648a31ddd028b37fa2f4d749afb69fcbccbf93c60"}
Sep 08 14:45:45.944019 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.944007    2583 scope.go:117] "RemoveContainer" containerID="bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b"
Sep 08 14:45:45.955764 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.955739    2583 scope.go:117] "RemoveContainer" containerID="bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b"
Sep 08 14:45:45.956152 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:45:45.956127    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b\": container with ID starting with bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b not found: ID does not exist" containerID="bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b"
Sep 08 14:45:45.956232 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.956163    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b"} err="failed to get container status \"bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b\": rpc error: code = NotFound desc = could not find container \"bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b\": container with ID starting with bd70624008408caa26ca8c09cebaf6af66fbb185b0e90c09155a0681f6a7a34b not found: ID does not exist"
Sep 08 14:45:45.962929 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.962903    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66bf74965-m4l4t"]
Sep 08 14:45:45.972082 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:45.972046    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66bf74965-m4l4t"]
Sep 08 14:45:46.709087 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:46.709045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:46.922986 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:46.922946    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:45:47.792711 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:45:47.792680    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="226a52ae-b256-43ac-9424-b9e5c7ca2b93" path="/var/lib/kubelet/pods/226a52ae-b256-43ac-9424-b9e5c7ca2b93/volumes"
Sep 08 14:46:02.251541 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:02.251472    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:02.365371 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:02.365333    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:02.528064 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:02.527967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:02.805083 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:02.804984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:03.507983 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:03.507935    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:03.673685 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:03.673625    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:03.972725 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:03.972691    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:05.602810 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:05.602772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:05.692820 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:05.692778    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:06.790944 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:06.790885    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:06.904066 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:06.904031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:07.086768 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.086666    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:07.654342 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.654313    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"]
Sep 08 14:46:07.654696 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.654684    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="226a52ae-b256-43ac-9424-b9e5c7ca2b93" containerName="authorino"
Sep 08 14:46:07.654742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.654699    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="226a52ae-b256-43ac-9424-b9e5c7ca2b93" containerName="authorino"
Sep 08 14:46:07.654776 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.654767    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="226a52ae-b256-43ac-9424-b9e5c7ca2b93" containerName="authorino"
Sep 08 14:46:07.657866 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.657849    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:46:07.662615 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.662591    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-798a5d-dockercfg-lvqpx\""
Sep 08 14:46:07.662819 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.662750    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-798a5d\""
Sep 08 14:46:07.669587 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.669565    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"]
Sep 08 14:46:07.717373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.717347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:07.728648 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.728627    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"]
Sep 08 14:46:07.732389 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.732341    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:46:07.745873 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.745848    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"]
Sep 08 14:46:07.785043 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.785018    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp8zq\" (UniqueName: \"kubernetes.io/projected/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-kube-api-access-dp8zq\") pod \"payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr\" (UID: \"64c2b1e9-e3cb-4a65-b016-5e59d1cef488\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:46:07.785170 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.785064    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr\" (UID: \"64c2b1e9-e3cb-4a65-b016-5e59d1cef488\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:46:07.785170 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.785091    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g2k6\" (UniqueName: \"kubernetes.io/projected/d169e6eb-618c-4fb6-b214-164a0a6ffb68-kube-api-access-2g2k6\") pod \"payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw\" (UID: \"d169e6eb-618c-4fb6-b214-164a0a6ffb68\") " pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:46:07.785253 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.785173    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d169e6eb-618c-4fb6-b214-164a0a6ffb68-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw\" (UID: \"d169e6eb-618c-4fb6-b214-164a0a6ffb68\") " pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:46:07.886104 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.886078    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp8zq\" (UniqueName: \"kubernetes.io/projected/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-kube-api-access-dp8zq\") pod \"payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr\" (UID: \"64c2b1e9-e3cb-4a65-b016-5e59d1cef488\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:46:07.886547 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.886118    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr\" (UID: \"64c2b1e9-e3cb-4a65-b016-5e59d1cef488\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:46:07.886547 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.886138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g2k6\" (UniqueName: \"kubernetes.io/projected/d169e6eb-618c-4fb6-b214-164a0a6ffb68-kube-api-access-2g2k6\") pod \"payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw\" (UID: \"d169e6eb-618c-4fb6-b214-164a0a6ffb68\") " pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:46:07.886547 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.886181    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d169e6eb-618c-4fb6-b214-164a0a6ffb68-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw\" (UID: \"d169e6eb-618c-4fb6-b214-164a0a6ffb68\") " pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:46:07.886976 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.886954    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d169e6eb-618c-4fb6-b214-164a0a6ffb68-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw\" (UID: \"d169e6eb-618c-4fb6-b214-164a0a6ffb68\") " pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:46:07.887061 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.886959    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr\" (UID: \"64c2b1e9-e3cb-4a65-b016-5e59d1cef488\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:46:07.896271 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.896250    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g2k6\" (UniqueName: \"kubernetes.io/projected/d169e6eb-618c-4fb6-b214-164a0a6ffb68-kube-api-access-2g2k6\") pod \"payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw\" (UID: \"d169e6eb-618c-4fb6-b214-164a0a6ffb68\") " pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:46:07.896375 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.896336    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp8zq\" (UniqueName: \"kubernetes.io/projected/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-kube-api-access-dp8zq\") pod \"payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr\" (UID: \"64c2b1e9-e3cb-4a65-b016-5e59d1cef488\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:46:07.969008 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:07.968731    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:46:08.042493 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:08.042405    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:46:08.095571 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:08.095530    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:08.275290 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:08.275267    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"]
Sep 08 14:46:08.276193 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:46:08.276168    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c2b1e9_e3cb_4a65_b016_5e59d1cef488.slice/crio-9f29bd1a5ae90236d34283fa3951bbb1e5d6506ed6fcbb744224ad86a68f4b97 WatchSource:0}: Error finding container 9f29bd1a5ae90236d34283fa3951bbb1e5d6506ed6fcbb744224ad86a68f4b97: Status 404 returned error can't find the container with id 9f29bd1a5ae90236d34283fa3951bbb1e5d6506ed6fcbb744224ad86a68f4b97
Sep 08 14:46:08.348124 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:08.348098    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"]
Sep 08 14:46:08.349094 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:46:08.349066    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd169e6eb_618c_4fb6_b214_164a0a6ffb68.slice/crio-0774ffb46664cad5ceb108165a7a7bc28b9955a4464b6c6a01d97bdaf573d306 WatchSource:0}: Error finding container 0774ffb46664cad5ceb108165a7a7bc28b9955a4464b6c6a01d97bdaf573d306: Status 404 returned error can't find the container with id 0774ffb46664cad5ceb108165a7a7bc28b9955a4464b6c6a01d97bdaf573d306
Sep 08 14:46:09.053232 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:09.053175    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw" event={"ID":"d169e6eb-618c-4fb6-b214-164a0a6ffb68","Type":"ContainerStarted","Data":"bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40"}
Sep 08 14:46:09.053695 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:09.053259    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:46:09.053695 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:09.053286    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw" event={"ID":"d169e6eb-618c-4fb6-b214-164a0a6ffb68","Type":"ContainerStarted","Data":"0774ffb46664cad5ceb108165a7a7bc28b9955a4464b6c6a01d97bdaf573d306"}
Sep 08 14:46:09.054756 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:09.054726    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr" event={"ID":"64c2b1e9-e3cb-4a65-b016-5e59d1cef488","Type":"ContainerStarted","Data":"a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826"}
Sep 08 14:46:09.054892 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:09.054779    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr" event={"ID":"64c2b1e9-e3cb-4a65-b016-5e59d1cef488","Type":"ContainerStarted","Data":"9f29bd1a5ae90236d34283fa3951bbb1e5d6506ed6fcbb744224ad86a68f4b97"}
Sep 08 14:46:09.054892 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:09.054802    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:46:09.076640 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:09.076596    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw" podStartSLOduration=2.076555389 podStartE2EDuration="2.076555389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:46:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:46:09.075320927 +0000 UTC m=+719.803137943" watchObservedRunningTime="2026-09-08 14:46:09.076555389 +0000 UTC m=+719.804372404"
Sep 08 14:46:09.110790 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:09.110742    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr" podStartSLOduration=2.110728343 podStartE2EDuration="2.110728343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:46:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:46:09.109278027 +0000 UTC m=+719.837095043" watchObservedRunningTime="2026-09-08 14:46:09.110728343 +0000 UTC m=+719.838545357"
Sep 08 14:46:10.424059 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:10.424025    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:10.510908 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:10.510877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:10.659190 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:10.659157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:10.974398 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:10.974368    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:11.089004 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:11.088967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:11.174953 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:11.174921    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:11.393621 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:11.393547    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:11.746428 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:11.746387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:13.169264 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:13.169221    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:13.252635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:13.252593    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:14.322624 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:14.322590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:14.429811 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:14.429784    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:14.832268 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:14.832230    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:14.947478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:14.947449    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:15.015350 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.015319    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"]
Sep 08 14:46:15.020020 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.020003    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:15.023504 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.023485    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3af37e0c\""
Sep 08 14:46:15.034460 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.034437    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"]
Sep 08 14:46:15.044219 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.044198    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:15.132348 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.132265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:15.149250 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.149218    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9cff434-1226-48a9-954c-03a549534703-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4\" (UID: \"d9cff434-1226-48a9-954c-03a549534703\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:15.149432 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.149285    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbwfn\" (UniqueName: \"kubernetes.io/projected/d9cff434-1226-48a9-954c-03a549534703-kube-api-access-bbwfn\") pod \"payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4\" (UID: \"d9cff434-1226-48a9-954c-03a549534703\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:15.170625 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.170599    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"]
Sep 08 14:46:15.174092 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.174072    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:15.177604 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.177588    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-3af37e0c-dockercfg-6gb8w\""
Sep 08 14:46:15.191031 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.191009    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"]
Sep 08 14:46:15.250581 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.250550    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9cff434-1226-48a9-954c-03a549534703-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4\" (UID: \"d9cff434-1226-48a9-954c-03a549534703\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:15.250725 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.250588    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd02afe5-6f42-459e-befc-275b3bfea3d2-plugins-config-volume\") pod \"payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5\" (UID: \"bd02afe5-6f42-459e-befc-275b3bfea3d2\") " pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:15.250725 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.250617    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjfhf\" (UniqueName: \"kubernetes.io/projected/bd02afe5-6f42-459e-befc-275b3bfea3d2-kube-api-access-mjfhf\") pod \"payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5\" (UID: \"bd02afe5-6f42-459e-befc-275b3bfea3d2\") " pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:15.250725 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.250637    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbwfn\" (UniqueName: \"kubernetes.io/projected/d9cff434-1226-48a9-954c-03a549534703-kube-api-access-bbwfn\") pod \"payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4\" (UID: \"d9cff434-1226-48a9-954c-03a549534703\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:15.251324 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.251305    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9cff434-1226-48a9-954c-03a549534703-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4\" (UID: \"d9cff434-1226-48a9-954c-03a549534703\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:15.262974 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.262947    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbwfn\" (UniqueName: \"kubernetes.io/projected/d9cff434-1226-48a9-954c-03a549534703-kube-api-access-bbwfn\") pod \"payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4\" (UID: \"d9cff434-1226-48a9-954c-03a549534703\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:15.330116 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.330074    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:15.351561 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.351535    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd02afe5-6f42-459e-befc-275b3bfea3d2-plugins-config-volume\") pod \"payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5\" (UID: \"bd02afe5-6f42-459e-befc-275b3bfea3d2\") " pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:15.351686 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.351587    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjfhf\" (UniqueName: \"kubernetes.io/projected/bd02afe5-6f42-459e-befc-275b3bfea3d2-kube-api-access-mjfhf\") pod \"payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5\" (UID: \"bd02afe5-6f42-459e-befc-275b3bfea3d2\") " pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:15.352350 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.352277    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd02afe5-6f42-459e-befc-275b3bfea3d2-plugins-config-volume\") pod \"payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5\" (UID: \"bd02afe5-6f42-459e-befc-275b3bfea3d2\") " pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:15.363813 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.363792    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjfhf\" (UniqueName: \"kubernetes.io/projected/bd02afe5-6f42-459e-befc-275b3bfea3d2-kube-api-access-mjfhf\") pod \"payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5\" (UID: \"bd02afe5-6f42-459e-befc-275b3bfea3d2\") " pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:15.484102 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.484070    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:15.605858 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.605813    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"]
Sep 08 14:46:15.607775 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:46:15.607752    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9cff434_1226_48a9_954c_03a549534703.slice/crio-6c00b09ace6427139a87b6cc05f0658b4e1fe3f0f45c8625095b08bb20aae750 WatchSource:0}: Error finding container 6c00b09ace6427139a87b6cc05f0658b4e1fe3f0f45c8625095b08bb20aae750: Status 404 returned error can't find the container with id 6c00b09ace6427139a87b6cc05f0658b4e1fe3f0f45c8625095b08bb20aae750
Sep 08 14:46:15.759522 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:15.759496    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"]
Sep 08 14:46:15.761353 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:46:15.761322    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd02afe5_6f42_459e_befc_275b3bfea3d2.slice/crio-765d187820ed8a5e88558191a0faefa88979e9fba4b3437e558591fe2a39c8ff WatchSource:0}: Error finding container 765d187820ed8a5e88558191a0faefa88979e9fba4b3437e558591fe2a39c8ff: Status 404 returned error can't find the container with id 765d187820ed8a5e88558191a0faefa88979e9fba4b3437e558591fe2a39c8ff
Sep 08 14:46:16.095446 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.095344    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5" event={"ID":"bd02afe5-6f42-459e-befc-275b3bfea3d2","Type":"ContainerStarted","Data":"78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560"}
Sep 08 14:46:16.095446 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.095406    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5" event={"ID":"bd02afe5-6f42-459e-befc-275b3bfea3d2","Type":"ContainerStarted","Data":"765d187820ed8a5e88558191a0faefa88979e9fba4b3437e558591fe2a39c8ff"}
Sep 08 14:46:16.095699 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.095555    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:16.097596 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.097567    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4" event={"ID":"d9cff434-1226-48a9-954c-03a549534703","Type":"ContainerStarted","Data":"461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06"}
Sep 08 14:46:16.097703 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.097614    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4" event={"ID":"d9cff434-1226-48a9-954c-03a549534703","Type":"ContainerStarted","Data":"6c00b09ace6427139a87b6cc05f0658b4e1fe3f0f45c8625095b08bb20aae750"}
Sep 08 14:46:16.097703 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.097669    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:16.117409 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.117363    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5" podStartSLOduration=1.1173484949999999 podStartE2EDuration="1.117348495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:46:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:46:16.116610836 +0000 UTC m=+726.844427853" watchObservedRunningTime="2026-09-08 14:46:16.117348495 +0000 UTC m=+726.845165508"
Sep 08 14:46:16.140349 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.140298    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4" podStartSLOduration=2.140278732 podStartE2EDuration="2.140278732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:46:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:46:16.138796155 +0000 UTC m=+726.866613170" watchObservedRunningTime="2026-09-08 14:46:16.140278732 +0000 UTC m=+726.868095747"
Sep 08 14:46:16.267145 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.267109    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:16.412121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.412014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:16.529264 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.529229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:16.802295 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.802266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:16.898727 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:16.898665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:19.949555 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:19.949520    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"]
Sep 08 14:46:19.953274 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:19.953252    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:19.956379 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:19.956356    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-c8c0e893-kserve-self-signed-certs\""
Sep 08 14:46:19.963498 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:19.963473    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"]
Sep 08 14:46:20.061449 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.061424    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:46:20.061599 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.061487    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:46:20.095164 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.095133    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-home\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.095164 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.095166    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tmp-dir\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.095407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.095184    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-model-cache\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.095407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.095322    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-dshm\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.095407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.095348    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2dbl\" (UniqueName: \"kubernetes.io/projected/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-kube-api-access-k2dbl\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.095407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.095372    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tls-certs\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.196693 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.196661    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-dshm\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.196693 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.196695    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2dbl\" (UniqueName: \"kubernetes.io/projected/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-kube-api-access-k2dbl\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.196949 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.196714    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tls-certs\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.197010 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.196973    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-home\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.197081 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.197010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tmp-dir\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.197081 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.197038    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-model-cache\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.197329 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.197304    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-home\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.197510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.197465    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-model-cache\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.197674 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.197491    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tmp-dir\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.199184 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.199163    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-dshm\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.199368 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.199354    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tls-certs\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.206881 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.206792    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2dbl\" (UniqueName: \"kubernetes.io/projected/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-kube-api-access-k2dbl\") pod \"e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") " pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.259704 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.259673    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:20.263696 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.263675    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:20.537566 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:20.537533    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"]
Sep 08 14:46:20.538960 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:46:20.538938    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7f5796e_e983_4bd1_b9e0_fb6180ffe219.slice/crio-da33ce55242a5c7ff3de7b50a9713289695ae77300f2c94f88cd4b15b6d8d650 WatchSource:0}: Error finding container da33ce55242a5c7ff3de7b50a9713289695ae77300f2c94f88cd4b15b6d8d650: Status 404 returned error can't find the container with id da33ce55242a5c7ff3de7b50a9713289695ae77300f2c94f88cd4b15b6d8d650
Sep 08 14:46:21.126692 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:21.126656    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9" event={"ID":"a7f5796e-e983-4bd1-b9e0-fb6180ffe219","Type":"ContainerStarted","Data":"dd08c6654f141633a78d57932075084037e8b39739b190db48e4b5e6be5d075b"}
Sep 08 14:46:21.127210 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:21.126701    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9" event={"ID":"a7f5796e-e983-4bd1-b9e0-fb6180ffe219","Type":"ContainerStarted","Data":"da33ce55242a5c7ff3de7b50a9713289695ae77300f2c94f88cd4b15b6d8d650"}
Sep 08 14:46:21.149775 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:21.149726    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9" podStartSLOduration=2.149710724 podStartE2EDuration="2.149710724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:46:21.148223934 +0000 UTC m=+731.876040948" watchObservedRunningTime="2026-09-08 14:46:21.149710724 +0000 UTC m=+731.877527737"
Sep 08 14:46:21.180157 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:21.180133    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:21.352018 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:21.351989    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:21.447072 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:21.446992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:21.547217 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:21.547185    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:22.991895 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:22.991862    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:23.090344 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:23.090310    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:23.213263 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:23.213231    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:24.278980 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:24.278949    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:25.914113 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:25.914035    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:27.104497 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:27.104465    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:27.105006 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:27.104539    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:28.114049 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:28.114018    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:28.743749 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:28.743702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:30.108466 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:30.108426    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:30.211051 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:30.211018    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:30.264909 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:30.264869    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:30.265060 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:30.264932    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:30.277284 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:30.277262    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:30.303334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:30.303307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:30.450370 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:30.450284    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:30.622890 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:30.622826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:31.089945 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:31.089911    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:31.193303 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:31.193276    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:46:31.327437 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:31.327405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:32.177966 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:32.177931    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:34.852267 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:34.852231    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:35.228762 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:35.228729    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:35.471800 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:35.471766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:36.538518 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:36.538491    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:36.630917 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:36.630888    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:36.860989 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:36.860902    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:37.142818 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.142724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:37.246735 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.246704    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"]
Sep 08 14:46:37.250395 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.250378    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:46:37.253633 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.253611    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-02a199-dockercfg-twlcz\""
Sep 08 14:46:37.253795 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.253667    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-02a199\""
Sep 08 14:46:37.269690 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.269665    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3b9ccdd-97a5-4095-a0d9-c299350db2be-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5\" (UID: \"f3b9ccdd-97a5-4095-a0d9-c299350db2be\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:46:37.269821 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.269707    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgfv5\" (UniqueName: \"kubernetes.io/projected/f3b9ccdd-97a5-4095-a0d9-c299350db2be-kube-api-access-rgfv5\") pod \"payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5\" (UID: \"f3b9ccdd-97a5-4095-a0d9-c299350db2be\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:46:37.270649 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.270631    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"]
Sep 08 14:46:37.370920 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.370889    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3b9ccdd-97a5-4095-a0d9-c299350db2be-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5\" (UID: \"f3b9ccdd-97a5-4095-a0d9-c299350db2be\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:46:37.371075 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.370927    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgfv5\" (UniqueName: \"kubernetes.io/projected/f3b9ccdd-97a5-4095-a0d9-c299350db2be-kube-api-access-rgfv5\") pod \"payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5\" (UID: \"f3b9ccdd-97a5-4095-a0d9-c299350db2be\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:46:37.371543 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.371524    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3b9ccdd-97a5-4095-a0d9-c299350db2be-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5\" (UID: \"f3b9ccdd-97a5-4095-a0d9-c299350db2be\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:46:37.380776 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.380744    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgfv5\" (UniqueName: \"kubernetes.io/projected/f3b9ccdd-97a5-4095-a0d9-c299350db2be-kube-api-access-rgfv5\") pod \"payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5\" (UID: \"f3b9ccdd-97a5-4095-a0d9-c299350db2be\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:46:37.472395 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.472368    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:37.561448 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.561421    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:46:37.606395 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.606359    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:37.837085 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:37.837060    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"]
Sep 08 14:46:37.838709 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:46:37.838684    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3b9ccdd_97a5_4095_a0d9_c299350db2be.slice/crio-fbe316f52dacaf5f08d6d4b6397c1c91709e5699e2579b82763c8c455d7a1759 WatchSource:0}: Error finding container fbe316f52dacaf5f08d6d4b6397c1c91709e5699e2579b82763c8c455d7a1759: Status 404 returned error can't find the container with id fbe316f52dacaf5f08d6d4b6397c1c91709e5699e2579b82763c8c455d7a1759
Sep 08 14:46:38.220813 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:38.220778    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5" event={"ID":"f3b9ccdd-97a5-4095-a0d9-c299350db2be","Type":"ContainerStarted","Data":"41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef"}
Sep 08 14:46:38.221025 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:38.220851    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5" event={"ID":"f3b9ccdd-97a5-4095-a0d9-c299350db2be","Type":"ContainerStarted","Data":"fbe316f52dacaf5f08d6d4b6397c1c91709e5699e2579b82763c8c455d7a1759"}
Sep 08 14:46:38.221025 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:38.221001    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:46:38.241159 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:38.241115    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5" podStartSLOduration=1.2411006310000001 podStartE2EDuration="1.241100631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:46:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:46:38.238394048 +0000 UTC m=+748.966211062" watchObservedRunningTime="2026-09-08 14:46:38.241100631 +0000 UTC m=+748.968917646"
Sep 08 14:46:38.343722 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:38.343689    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:38.451221 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:38.451185    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:39.427289 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:39.427259    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:39.668229 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:39.668196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:40.306944 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:40.306910    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:41.235280 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:41.235226    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:44.734126 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:44.734098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:45.387456 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.387370    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:45.412698 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.412666    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"]
Sep 08 14:46:45.412948 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.412925    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5" podUID="bd02afe5-6f42-459e-befc-275b3bfea3d2" containerName="payload-processing" containerID="cri-o://78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560" gracePeriod=30
Sep 08 14:46:45.423734 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.423707    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"]
Sep 08 14:46:45.424009 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.423969    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4" podUID="d9cff434-1226-48a9-954c-03a549534703" containerName="payload-pre-processing" containerID="cri-o://461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06" gracePeriod=30
Sep 08 14:46:45.488086 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.488058    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:45.591348 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.591320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:45.690743 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.690717    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:45.702276 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.702251    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:45.710885 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.710863    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:45.747526 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.747493    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9cff434-1226-48a9-954c-03a549534703-plugins-config-volume\") pod \"d9cff434-1226-48a9-954c-03a549534703\" (UID: \"d9cff434-1226-48a9-954c-03a549534703\") "
Sep 08 14:46:45.747899 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.747550    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd02afe5-6f42-459e-befc-275b3bfea3d2-plugins-config-volume\") pod \"bd02afe5-6f42-459e-befc-275b3bfea3d2\" (UID: \"bd02afe5-6f42-459e-befc-275b3bfea3d2\") "
Sep 08 14:46:45.747899 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.747601    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbwfn\" (UniqueName: \"kubernetes.io/projected/d9cff434-1226-48a9-954c-03a549534703-kube-api-access-bbwfn\") pod \"d9cff434-1226-48a9-954c-03a549534703\" (UID: \"d9cff434-1226-48a9-954c-03a549534703\") "
Sep 08 14:46:45.747899 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.747638    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjfhf\" (UniqueName: \"kubernetes.io/projected/bd02afe5-6f42-459e-befc-275b3bfea3d2-kube-api-access-mjfhf\") pod \"bd02afe5-6f42-459e-befc-275b3bfea3d2\" (UID: \"bd02afe5-6f42-459e-befc-275b3bfea3d2\") "
Sep 08 14:46:45.748053 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.747998    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd02afe5-6f42-459e-befc-275b3bfea3d2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bd02afe5-6f42-459e-befc-275b3bfea3d2" (UID: "bd02afe5-6f42-459e-befc-275b3bfea3d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:46:45.748202 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.748172    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9cff434-1226-48a9-954c-03a549534703-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d9cff434-1226-48a9-954c-03a549534703" (UID: "d9cff434-1226-48a9-954c-03a549534703"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:46:45.750197 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.750169    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9cff434-1226-48a9-954c-03a549534703-kube-api-access-bbwfn" (OuterVolumeSpecName: "kube-api-access-bbwfn") pod "d9cff434-1226-48a9-954c-03a549534703" (UID: "d9cff434-1226-48a9-954c-03a549534703"). InnerVolumeSpecName "kube-api-access-bbwfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:46:45.750322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.750300    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd02afe5-6f42-459e-befc-275b3bfea3d2-kube-api-access-mjfhf" (OuterVolumeSpecName: "kube-api-access-mjfhf") pod "bd02afe5-6f42-459e-befc-275b3bfea3d2" (UID: "bd02afe5-6f42-459e-befc-275b3bfea3d2"). InnerVolumeSpecName "kube-api-access-mjfhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:46:45.784090 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.784056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:45.849066 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.849038    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9cff434-1226-48a9-954c-03a549534703-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:46:45.849066 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.849064    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd02afe5-6f42-459e-befc-275b3bfea3d2-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:46:45.849225 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.849073    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbwfn\" (UniqueName: \"kubernetes.io/projected/d9cff434-1226-48a9-954c-03a549534703-kube-api-access-bbwfn\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:46:45.849225 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:45.849083    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjfhf\" (UniqueName: \"kubernetes.io/projected/bd02afe5-6f42-459e-befc-275b3bfea3d2-kube-api-access-mjfhf\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:46:46.262136 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.262103    2583 generic.go:356] "Generic (PLEG): container finished" podID="d9cff434-1226-48a9-954c-03a549534703" containerID="461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06" exitCode=0
Sep 08 14:46:46.262307 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.262144    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"
Sep 08 14:46:46.262307 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.262183    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4" event={"ID":"d9cff434-1226-48a9-954c-03a549534703","Type":"ContainerDied","Data":"461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06"}
Sep 08 14:46:46.262307 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.262237    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4" event={"ID":"d9cff434-1226-48a9-954c-03a549534703","Type":"ContainerDied","Data":"6c00b09ace6427139a87b6cc05f0658b4e1fe3f0f45c8625095b08bb20aae750"}
Sep 08 14:46:46.262307 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.262261    2583 scope.go:117] "RemoveContainer" containerID="461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06"
Sep 08 14:46:46.263633 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.263613    2583 generic.go:356] "Generic (PLEG): container finished" podID="bd02afe5-6f42-459e-befc-275b3bfea3d2" containerID="78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560" exitCode=0
Sep 08 14:46:46.263964 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.263648    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5" event={"ID":"bd02afe5-6f42-459e-befc-275b3bfea3d2","Type":"ContainerDied","Data":"78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560"}
Sep 08 14:46:46.263964 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.263653    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"
Sep 08 14:46:46.263964 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.263676    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5" event={"ID":"bd02afe5-6f42-459e-befc-275b3bfea3d2","Type":"ContainerDied","Data":"765d187820ed8a5e88558191a0faefa88979e9fba4b3437e558591fe2a39c8ff"}
Sep 08 14:46:46.271039 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.271018    2583 scope.go:117] "RemoveContainer" containerID="461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06"
Sep 08 14:46:46.271302 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:46:46.271284    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06\": container with ID starting with 461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06 not found: ID does not exist" containerID="461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06"
Sep 08 14:46:46.271341 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.271312    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06"} err="failed to get container status \"461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06\": rpc error: code = NotFound desc = could not find container \"461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06\": container with ID starting with 461d43c9e118f83ff233921d9468cb1a8fbdada3278b7ed06cbc908e45868e06 not found: ID does not exist"
Sep 08 14:46:46.271341 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.271330    2583 scope.go:117] "RemoveContainer" containerID="78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560"
Sep 08 14:46:46.279210 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.279178    2583 scope.go:117] "RemoveContainer" containerID="78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560"
Sep 08 14:46:46.279616 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:46:46.279590    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560\": container with ID starting with 78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560 not found: ID does not exist" containerID="78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560"
Sep 08 14:46:46.279709 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.279625    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560"} err="failed to get container status \"78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560\": rpc error: code = NotFound desc = could not find container \"78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560\": container with ID starting with 78180392df8be9c89fd75aec4ab3a456630e62f6c153cac804ed0e18d9a95560 not found: ID does not exist"
Sep 08 14:46:46.314003 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.313967    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"]
Sep 08 14:46:46.319921 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.319895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:46.325962 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.325939    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3af37e0c-68d48ff86-d7dt4"]
Sep 08 14:46:46.362071 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.362042    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"]
Sep 08 14:46:46.373484 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.373458    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3af37e0c-7dd85d9d6c-gjjk5"]
Sep 08 14:46:46.548706 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.548628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:46.646239 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.646209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:46.825899 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:46.825799    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:47.155250 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:47.155168    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:47.793190 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:47.793159    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd02afe5-6f42-459e-befc-275b3bfea3d2" path="/var/lib/kubelet/pods/bd02afe5-6f42-459e-befc-275b3bfea3d2/volumes"
Sep 08 14:46:47.793527 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:47.793515    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9cff434-1226-48a9-954c-03a549534703" path="/var/lib/kubelet/pods/d9cff434-1226-48a9-954c-03a549534703/volumes"
Sep 08 14:46:49.233892 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:49.233858    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:46:52.009596 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:52.009525    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:52.111723 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:52.111692    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:52.323156 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:52.323077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:52.755998 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:52.755963    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:52.855344 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:52.855315    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:52.970943 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:52.970900    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:53.067887 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.067763    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:53.117491 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.117460    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"]
Sep 08 14:46:53.118078 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.118062    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd02afe5-6f42-459e-befc-275b3bfea3d2" containerName="payload-processing"
Sep 08 14:46:53.118120 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.118081    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd02afe5-6f42-459e-befc-275b3bfea3d2" containerName="payload-processing"
Sep 08 14:46:53.118120 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.118102    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9cff434-1226-48a9-954c-03a549534703" containerName="payload-pre-processing"
Sep 08 14:46:53.118120 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.118108    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9cff434-1226-48a9-954c-03a549534703" containerName="payload-pre-processing"
Sep 08 14:46:53.118218 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.118170    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd02afe5-6f42-459e-befc-275b3bfea3d2" containerName="payload-processing"
Sep 08 14:46:53.118218 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.118183    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9cff434-1226-48a9-954c-03a549534703" containerName="payload-pre-processing"
Sep 08 14:46:53.122463 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.122448    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:46:53.126162 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.126139    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-8ac49eaa-dockercfg-t8vww\""
Sep 08 14:46:53.126461 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.126441    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-8ac49eaa\""
Sep 08 14:46:53.134208 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.134187    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"]
Sep 08 14:46:53.216749 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.216713    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c\" (UID: \"a9aa5146-de05-4b05-9ff8-1e4ceff1c981\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:46:53.216749 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.216751    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxmm8\" (UniqueName: \"kubernetes.io/projected/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-kube-api-access-lxmm8\") pod \"payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c\" (UID: \"a9aa5146-de05-4b05-9ff8-1e4ceff1c981\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:46:53.318075 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.318000    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c\" (UID: \"a9aa5146-de05-4b05-9ff8-1e4ceff1c981\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:46:53.318075 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.318033    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxmm8\" (UniqueName: \"kubernetes.io/projected/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-kube-api-access-lxmm8\") pod \"payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c\" (UID: \"a9aa5146-de05-4b05-9ff8-1e4ceff1c981\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:46:53.318683 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.318663    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c\" (UID: \"a9aa5146-de05-4b05-9ff8-1e4ceff1c981\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:46:53.328685 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.328658    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxmm8\" (UniqueName: \"kubernetes.io/projected/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-kube-api-access-lxmm8\") pod \"payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c\" (UID: \"a9aa5146-de05-4b05-9ff8-1e4ceff1c981\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:46:53.432743 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.432711    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:46:53.713300 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:53.713271    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"]
Sep 08 14:46:53.713889 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:46:53.713863    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9aa5146_de05_4b05_9ff8_1e4ceff1c981.slice/crio-f2c2681e97bd90d44c55abdf188ca3fa7e9e0b34587e195d6c6a6d18e7b7db0e WatchSource:0}: Error finding container f2c2681e97bd90d44c55abdf188ca3fa7e9e0b34587e195d6c6a6d18e7b7db0e: Status 404 returned error can't find the container with id f2c2681e97bd90d44c55abdf188ca3fa7e9e0b34587e195d6c6a6d18e7b7db0e
Sep 08 14:46:54.303700 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:54.303661    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c" event={"ID":"a9aa5146-de05-4b05-9ff8-1e4ceff1c981","Type":"ContainerStarted","Data":"9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0"}
Sep 08 14:46:54.304155 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:54.303716    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c" event={"ID":"a9aa5146-de05-4b05-9ff8-1e4ceff1c981","Type":"ContainerStarted","Data":"f2c2681e97bd90d44c55abdf188ca3fa7e9e0b34587e195d6c6a6d18e7b7db0e"}
Sep 08 14:46:54.304155 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:54.303790    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:46:54.327589 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:54.327542    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c" podStartSLOduration=1.327526471 podStartE2EDuration="1.327526471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:46:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:46:54.3268141 +0000 UTC m=+765.054631127" watchObservedRunningTime="2026-09-08 14:46:54.327526471 +0000 UTC m=+765.055343520"
Sep 08 14:46:57.532868 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:57.532812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:46:59.748513 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:46:59.748477    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:05.310603 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:05.310570    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:47:10.674179 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.674145    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"]
Sep 08 14:47:10.679550 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.679523    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.683019 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.683001    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-c8c0e893-kserve-self-signed-certs\""
Sep 08 14:47:10.689029 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.688997    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"]
Sep 08 14:47:10.731526 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.731497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:10.783000 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.782963    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tmp-dir\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.783153 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.783004    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tls-certs\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.783153 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.783081    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mknd8\" (UniqueName: \"kubernetes.io/projected/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-kube-api-access-mknd8\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.783244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.783200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-model-cache\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.783244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.783235    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-home\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.783331 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.783260    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-dshm\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.884110 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.884072    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-home\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.884294 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.884115    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-dshm\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.884294 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.884167    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tmp-dir\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.884294 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.884195    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tls-certs\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.884294 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.884261    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mknd8\" (UniqueName: \"kubernetes.io/projected/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-kube-api-access-mknd8\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.884519 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.884401    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-model-cache\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.884996 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.884973    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-home\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.885125 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.885100    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tmp-dir\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.885193 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.885130    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-model-cache\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.886863 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.886824    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-dshm\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.887252 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.887228    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tls-certs\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.894222 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.894194    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mknd8\" (UniqueName: \"kubernetes.io/projected/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-kube-api-access-mknd8\") pod \"e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") " pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:10.988782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:10.988754    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:11.261677 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:11.261650    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"]
Sep 08 14:47:11.394821 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:11.394779    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz" event={"ID":"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d","Type":"ContainerStarted","Data":"37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de"}
Sep 08 14:47:11.394994 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:11.394857    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz" event={"ID":"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d","Type":"ContainerStarted","Data":"3139b0796e8da2014d6a4a375fe8a19a33b53377c0dc23213058697090e32f47"}
Sep 08 14:47:11.429367 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:11.429320    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz" podStartSLOduration=1.429305401 podStartE2EDuration="1.429305401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:47:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:47:11.427697085 +0000 UTC m=+782.155514099" watchObservedRunningTime="2026-09-08 14:47:11.429305401 +0000 UTC m=+782.157122414"
Sep 08 14:47:13.381217 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:13.381183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:17.479609 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.479576    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"]
Sep 08 14:47:17.483606 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.483585    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.486698 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.486678    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-71518b\"/\"openshift-service-ca.crt\""
Sep 08 14:47:17.487996 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.487778    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-71518b\"/\"kube-root-ca.crt\""
Sep 08 14:47:17.487996 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.487855    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-71518b\"/\"autores-model-71518b-kserve-self-signed-certs\""
Sep 08 14:47:17.488167 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.488107    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-71518b\"/\"default-dockercfg-c5kh9\""
Sep 08 14:47:17.490476 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.490454    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"]
Sep 08 14:47:17.507848 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.507785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:17.648970 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.648934    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-home\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.649154 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.648980    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdn2b\" (UniqueName: \"kubernetes.io/projected/1163899b-7096-4f22-b7e6-943b85e19b89-kube-api-access-qdn2b\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.649154 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.649044    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-model-cache\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.649154 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.649108    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-tmp-dir\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.649154 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.649143    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1163899b-7096-4f22-b7e6-943b85e19b89-tls-certs\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.649311 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.649197    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-dshm\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.750383 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.750297    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-model-cache\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.750383 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.750351    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-tmp-dir\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.750602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.750387    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1163899b-7096-4f22-b7e6-943b85e19b89-tls-certs\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.750602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.750418    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-dshm\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.750602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.750438    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-home\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.750602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.750480    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdn2b\" (UniqueName: \"kubernetes.io/projected/1163899b-7096-4f22-b7e6-943b85e19b89-kube-api-access-qdn2b\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.750814 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.750761    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-model-cache\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.750885 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.750813    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-tmp-dir\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.750885 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.750853    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-home\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.752735 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.752717    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-dshm\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.753147 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.753126    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1163899b-7096-4f22-b7e6-943b85e19b89-tls-certs\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.761017 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.760995    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdn2b\" (UniqueName: \"kubernetes.io/projected/1163899b-7096-4f22-b7e6-943b85e19b89-kube-api-access-qdn2b\") pod \"autores-model-71518b-kserve-84d776884c-vlwfk\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") " pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:17.793988 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:17.793970    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:18.070403 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:18.070375    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"]
Sep 08 14:47:18.072771 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:47:18.072741    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1163899b_7096_4f22_b7e6_943b85e19b89.slice/crio-13944acacfa59b0380ea5c84e4064fba83fb15ee21831bc2a084222d3253e335 WatchSource:0}: Error finding container 13944acacfa59b0380ea5c84e4064fba83fb15ee21831bc2a084222d3253e335: Status 404 returned error can't find the container with id 13944acacfa59b0380ea5c84e4064fba83fb15ee21831bc2a084222d3253e335
Sep 08 14:47:18.431607 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:18.431512    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk" event={"ID":"1163899b-7096-4f22-b7e6-943b85e19b89","Type":"ContainerStarted","Data":"28c45065f318148ab7f0fbe8c7260ef4f76d64cdbfc42e1180a4120cc49603f1"}
Sep 08 14:47:18.431607 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:18.431560    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk" event={"ID":"1163899b-7096-4f22-b7e6-943b85e19b89","Type":"ContainerStarted","Data":"13944acacfa59b0380ea5c84e4064fba83fb15ee21831bc2a084222d3253e335"}
Sep 08 14:47:18.452474 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:18.452426    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk" podStartSLOduration=1.452411183 podStartE2EDuration="1.452411183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:47:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:47:18.450579837 +0000 UTC m=+789.178396851" watchObservedRunningTime="2026-09-08 14:47:18.452411183 +0000 UTC m=+789.180228196"
Sep 08 14:47:20.989682 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:20.989643    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:20.990106 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:20.989698    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:21.002650 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:21.002624    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:21.456817 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:21.456737    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:47:22.972967 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:22.972935    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:23.013111 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.013073    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"]
Sep 08 14:47:23.013382 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.013351    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c" podUID="a9aa5146-de05-4b05-9ff8-1e4ceff1c981" containerName="payload-pre-processing" containerID="cri-o://9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0" gracePeriod=30
Sep 08 14:47:23.072425 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.072396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:23.177817 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.177781    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:23.275574 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.275541    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:23.285934 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.285912    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:47:23.300895 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.300855    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-plugins-config-volume\") pod \"a9aa5146-de05-4b05-9ff8-1e4ceff1c981\" (UID: \"a9aa5146-de05-4b05-9ff8-1e4ceff1c981\") "
Sep 08 14:47:23.301067 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.300907    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxmm8\" (UniqueName: \"kubernetes.io/projected/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-kube-api-access-lxmm8\") pod \"a9aa5146-de05-4b05-9ff8-1e4ceff1c981\" (UID: \"a9aa5146-de05-4b05-9ff8-1e4ceff1c981\") "
Sep 08 14:47:23.301240 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.301219    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a9aa5146-de05-4b05-9ff8-1e4ceff1c981" (UID: "a9aa5146-de05-4b05-9ff8-1e4ceff1c981"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:47:23.303758 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.303734    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-kube-api-access-lxmm8" (OuterVolumeSpecName: "kube-api-access-lxmm8") pod "a9aa5146-de05-4b05-9ff8-1e4ceff1c981" (UID: "a9aa5146-de05-4b05-9ff8-1e4ceff1c981"). InnerVolumeSpecName "kube-api-access-lxmm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:47:23.393071 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.393040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:23.402402 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.402371    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:47:23.402402 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.402403    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxmm8\" (UniqueName: \"kubernetes.io/projected/a9aa5146-de05-4b05-9ff8-1e4ceff1c981-kube-api-access-lxmm8\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:47:23.458734 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.458701    2583 generic.go:356] "Generic (PLEG): container finished" podID="a9aa5146-de05-4b05-9ff8-1e4ceff1c981" containerID="9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0" exitCode=0
Sep 08 14:47:23.458917 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.458769    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"
Sep 08 14:47:23.458917 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.458773    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c" event={"ID":"a9aa5146-de05-4b05-9ff8-1e4ceff1c981","Type":"ContainerDied","Data":"9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0"}
Sep 08 14:47:23.458917 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.458901    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c" event={"ID":"a9aa5146-de05-4b05-9ff8-1e4ceff1c981","Type":"ContainerDied","Data":"f2c2681e97bd90d44c55abdf188ca3fa7e9e0b34587e195d6c6a6d18e7b7db0e"}
Sep 08 14:47:23.459082 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.458925    2583 scope.go:117] "RemoveContainer" containerID="9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0"
Sep 08 14:47:23.470351 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.470327    2583 scope.go:117] "RemoveContainer" containerID="9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0"
Sep 08 14:47:23.471030 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:47:23.470962    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0\": container with ID starting with 9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0 not found: ID does not exist" containerID="9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0"
Sep 08 14:47:23.471030 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.471003    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0"} err="failed to get container status \"9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0\": rpc error: code = NotFound desc = could not find container \"9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0\": container with ID starting with 9591ab7558bdbababfbec109533b7de43eb9e88142f8a4fb7e57492b358665f0 not found: ID does not exist"
Sep 08 14:47:23.488140 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.488110    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"]
Sep 08 14:47:23.494121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.494097    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8ac49eaa-fd97fd6f8-vpj8c"]
Sep 08 14:47:23.797453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.797417    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9aa5146-de05-4b05-9ff8-1e4ceff1c981" path="/var/lib/kubelet/pods/a9aa5146-de05-4b05-9ff8-1e4ceff1c981/volumes"
Sep 08 14:47:23.827709 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:23.827670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:24.198637 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:24.198552    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:24.455870 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:24.455761    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:26.560767 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:26.560734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:26.661625 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:26.661590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:27.697826 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:27.697799    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:27.794873 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:27.794840    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:27.794873 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:27.794881    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:27.803201 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:27.803176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:27.808001 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:27.807980    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:28.051299 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.051267    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:28.287627 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.287591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:28.386325 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.386230    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:28.435346 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.435315    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"]
Sep 08 14:47:28.435794 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.435778    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9aa5146-de05-4b05-9ff8-1e4ceff1c981" containerName="payload-pre-processing"
Sep 08 14:47:28.435876 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.435796    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9aa5146-de05-4b05-9ff8-1e4ceff1c981" containerName="payload-pre-processing"
Sep 08 14:47:28.435922 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.435892    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9aa5146-de05-4b05-9ff8-1e4ceff1c981" containerName="payload-pre-processing"
Sep 08 14:47:28.440401 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.440384    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:28.444212 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.444180    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-8066138e\""
Sep 08 14:47:28.444325 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.444308    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-8066138e-dockercfg-kkdwl\""
Sep 08 14:47:28.470077 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.470052    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"]
Sep 08 14:47:28.498431 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.498395    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:47:28.552973 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.552936    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkszz\" (UniqueName: \"kubernetes.io/projected/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-kube-api-access-dkszz\") pod \"payload-processing-e2e-ait-8066138e-64d98946df-lxb7n\" (UID: \"4911ea1a-62b6-48f2-bc4d-208cf813bb9b\") " pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:28.553142 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.553071    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-plugins-config-volume\") pod \"payload-processing-e2e-ait-8066138e-64d98946df-lxb7n\" (UID: \"4911ea1a-62b6-48f2-bc4d-208cf813bb9b\") " pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:28.654353 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.654277    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkszz\" (UniqueName: \"kubernetes.io/projected/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-kube-api-access-dkszz\") pod \"payload-processing-e2e-ait-8066138e-64d98946df-lxb7n\" (UID: \"4911ea1a-62b6-48f2-bc4d-208cf813bb9b\") " pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:28.654353 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.654334    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-plugins-config-volume\") pod \"payload-processing-e2e-ait-8066138e-64d98946df-lxb7n\" (UID: \"4911ea1a-62b6-48f2-bc4d-208cf813bb9b\") " pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:28.655045 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.655011    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-plugins-config-volume\") pod \"payload-processing-e2e-ait-8066138e-64d98946df-lxb7n\" (UID: \"4911ea1a-62b6-48f2-bc4d-208cf813bb9b\") " pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:28.664301 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.664273    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkszz\" (UniqueName: \"kubernetes.io/projected/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-kube-api-access-dkszz\") pod \"payload-processing-e2e-ait-8066138e-64d98946df-lxb7n\" (UID: \"4911ea1a-62b6-48f2-bc4d-208cf813bb9b\") " pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:28.751927 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:28.751898    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:29.026163 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:29.026135    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"]
Sep 08 14:47:29.027888 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:47:29.027862    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4911ea1a_62b6_48f2_bc4d_208cf813bb9b.slice/crio-a344b6b94158103ec20ebc6ed5884ec0931b8c2b5dadb7c30dab2f8d2f4dec07 WatchSource:0}: Error finding container a344b6b94158103ec20ebc6ed5884ec0931b8c2b5dadb7c30dab2f8d2f4dec07: Status 404 returned error can't find the container with id a344b6b94158103ec20ebc6ed5884ec0931b8c2b5dadb7c30dab2f8d2f4dec07
Sep 08 14:47:29.492785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:29.492750    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n" event={"ID":"4911ea1a-62b6-48f2-bc4d-208cf813bb9b","Type":"ContainerStarted","Data":"22ee3bc412faeec24f269cc7f84281e73f099a9133c3c8f915f427372b877ccc"}
Sep 08 14:47:29.492977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:29.492929    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:29.493024 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:29.492975    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n" event={"ID":"4911ea1a-62b6-48f2-bc4d-208cf813bb9b","Type":"ContainerStarted","Data":"a344b6b94158103ec20ebc6ed5884ec0931b8c2b5dadb7c30dab2f8d2f4dec07"}
Sep 08 14:47:29.511803 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:29.511758    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n" podStartSLOduration=1.511745247 podStartE2EDuration="1.511745247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:47:29.51110985 +0000 UTC m=+800.238926864" watchObservedRunningTime="2026-09-08 14:47:29.511745247 +0000 UTC m=+800.239562306"
Sep 08 14:47:40.500391 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:40.500318    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:49.602292 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:49.602258    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:51.662781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:51.662747    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"]
Sep 08 14:47:51.683513 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:51.683488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:52.471449 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:52.471418    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"]
Sep 08 14:47:52.476453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:52.476428    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"
Sep 08 14:47:52.479504 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:52.479446    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 14:47:52.479720 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:52.479705    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:47:52.491467 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:52.491440    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"]
Sep 08 14:47:52.513322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:52.513294    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"]
Sep 08 14:47:52.574365 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:52.574331    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:52.581551 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:52.581521    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b21602-f4f7-4b9f-ab4d-4c26548c8b49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-71518b-xn2bv\" (UID: \"75b21602-f4f7-4b9f-ab4d-4c26548c8b49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"
Sep 08 14:47:52.682667 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:52.682622    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b21602-f4f7-4b9f-ab4d-4c26548c8b49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-71518b-xn2bv\" (UID: \"75b21602-f4f7-4b9f-ab4d-4c26548c8b49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"
Sep 08 14:47:52.683249 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:52.683229    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b21602-f4f7-4b9f-ab4d-4c26548c8b49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-71518b-xn2bv\" (UID: \"75b21602-f4f7-4b9f-ab4d-4c26548c8b49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"
Sep 08 14:47:52.788339 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:52.788262    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"
Sep 08 14:47:53.065473 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:53.065447    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"]
Sep 08 14:47:53.068017 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:47:53.067978    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b21602_f4f7_4b9f_ab4d_4c26548c8b49.slice/crio-d5fb6291ba99c021f3255aabf2a9aebd09da55a511e67f428d72ba58e4cd2d01 WatchSource:0}: Error finding container d5fb6291ba99c021f3255aabf2a9aebd09da55a511e67f428d72ba58e4cd2d01: Status 404 returned error can't find the container with id d5fb6291ba99c021f3255aabf2a9aebd09da55a511e67f428d72ba58e4cd2d01
Sep 08 14:47:53.116761 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:53.116739    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:53.615809 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:53.615774    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv" event={"ID":"75b21602-f4f7-4b9f-ab4d-4c26548c8b49","Type":"ContainerStarted","Data":"d5fb6291ba99c021f3255aabf2a9aebd09da55a511e67f428d72ba58e4cd2d01"}
Sep 08 14:47:53.789063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:53.789022    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:54.591563 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:54.591171    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"]
Sep 08 14:47:54.616350 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:54.616320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:56.631265 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:56.631232    2583 generic.go:356] "Generic (PLEG): container finished" podID="75b21602-f4f7-4b9f-ab4d-4c26548c8b49" containerID="2764574200b764ebcfaf41cd32922f880113e2838ad3f2e2681c03e425597c42" exitCode=0
Sep 08 14:47:56.631680 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:56.631328    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv" event={"ID":"75b21602-f4f7-4b9f-ab4d-4c26548c8b49","Type":"ContainerDied","Data":"2764574200b764ebcfaf41cd32922f880113e2838ad3f2e2681c03e425597c42"}
Sep 08 14:47:57.776605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:57.776582    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"
Sep 08 14:47:57.833357 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:57.833330    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b21602-f4f7-4b9f-ab4d-4c26548c8b49-maas-api-service-ca\") pod \"75b21602-f4f7-4b9f-ab4d-4c26548c8b49\" (UID: \"75b21602-f4f7-4b9f-ab4d-4c26548c8b49\") "
Sep 08 14:47:57.833628 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:57.833608    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75b21602-f4f7-4b9f-ab4d-4c26548c8b49-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "75b21602-f4f7-4b9f-ab4d-4c26548c8b49" (UID: "75b21602-f4f7-4b9f-ab4d-4c26548c8b49"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:47:57.934361 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:57.934287    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b21602-f4f7-4b9f-ab4d-4c26548c8b49-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:47:58.491746 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.491707    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:58.527494 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.527460    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"]
Sep 08 14:47:58.527735 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.527711    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n" podUID="4911ea1a-62b6-48f2-bc4d-208cf813bb9b" containerName="payload-processing" containerID="cri-o://22ee3bc412faeec24f269cc7f84281e73f099a9133c3c8f915f427372b877ccc" gracePeriod=30
Sep 08 14:47:58.591374 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.591317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:58.643167 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.643138    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"
Sep 08 14:47:58.643321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.643165    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv" event={"ID":"75b21602-f4f7-4b9f-ab4d-4c26548c8b49","Type":"ContainerDied","Data":"d5fb6291ba99c021f3255aabf2a9aebd09da55a511e67f428d72ba58e4cd2d01"}
Sep 08 14:47:58.643321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.643226    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5fb6291ba99c021f3255aabf2a9aebd09da55a511e67f428d72ba58e4cd2d01"
Sep 08 14:47:58.654339 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.654305    2583 generic.go:356] "Generic (PLEG): container finished" podID="4911ea1a-62b6-48f2-bc4d-208cf813bb9b" containerID="22ee3bc412faeec24f269cc7f84281e73f099a9133c3c8f915f427372b877ccc" exitCode=0
Sep 08 14:47:58.654499 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.654355    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n" event={"ID":"4911ea1a-62b6-48f2-bc4d-208cf813bb9b","Type":"ContainerDied","Data":"22ee3bc412faeec24f269cc7f84281e73f099a9133c3c8f915f427372b877ccc"}
Sep 08 14:47:58.694610 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.694569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:58.790737 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.790707    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:47:58.803643 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.803619    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:58.843004 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.842973    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkszz\" (UniqueName: \"kubernetes.io/projected/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-kube-api-access-dkszz\") pod \"4911ea1a-62b6-48f2-bc4d-208cf813bb9b\" (UID: \"4911ea1a-62b6-48f2-bc4d-208cf813bb9b\") "
Sep 08 14:47:58.843165 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.843049    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-plugins-config-volume\") pod \"4911ea1a-62b6-48f2-bc4d-208cf813bb9b\" (UID: \"4911ea1a-62b6-48f2-bc4d-208cf813bb9b\") "
Sep 08 14:47:58.843501 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.843465    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4911ea1a-62b6-48f2-bc4d-208cf813bb9b" (UID: "4911ea1a-62b6-48f2-bc4d-208cf813bb9b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:47:58.845257 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.845228    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-kube-api-access-dkszz" (OuterVolumeSpecName: "kube-api-access-dkszz") pod "4911ea1a-62b6-48f2-bc4d-208cf813bb9b" (UID: "4911ea1a-62b6-48f2-bc4d-208cf813bb9b"). InnerVolumeSpecName "kube-api-access-dkszz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:47:58.944291 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.944256    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:47:58.944291 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:58.944286    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkszz\" (UniqueName: \"kubernetes.io/projected/4911ea1a-62b6-48f2-bc4d-208cf813bb9b-kube-api-access-dkszz\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:47:59.660235 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:59.660206    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n" event={"ID":"4911ea1a-62b6-48f2-bc4d-208cf813bb9b","Type":"ContainerDied","Data":"a344b6b94158103ec20ebc6ed5884ec0931b8c2b5dadb7c30dab2f8d2f4dec07"}
Sep 08 14:47:59.660235 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:59.660238    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"
Sep 08 14:47:59.660486 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:59.660260    2583 scope.go:117] "RemoveContainer" containerID="22ee3bc412faeec24f269cc7f84281e73f099a9133c3c8f915f427372b877ccc"
Sep 08 14:47:59.683776 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:59.683747    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"]
Sep 08 14:47:59.686921 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:59.686896    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8066138e-64d98946df-lxb7n"]
Sep 08 14:47:59.794386 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:59.794346    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4911ea1a-62b6-48f2-bc4d-208cf813bb9b" path="/var/lib/kubelet/pods/4911ea1a-62b6-48f2-bc4d-208cf813bb9b/volumes"
Sep 08 14:47:59.852812 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:47:59.852783    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:00.285399 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:00.285365    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:00.625152 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:00.625073    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:01.111236 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:01.111202    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:02.616516 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:02.616478    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:03.194198 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:03.194156    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:03.294481 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:03.294429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:03.559230 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:03.559192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:03.669175 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:03.669141    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:03.806008 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:03.805973    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:03.890146 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:03.890073    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"]
Sep 08 14:48:03.894350 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:03.894328    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-71518b-xn2bv"]
Sep 08 14:48:03.915588 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:03.915547    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:04.221182 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:04.221138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:04.377880 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:04.377823    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:04.468736 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:04.468677    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:04.719496 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:04.719462    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:04.884685 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:04.884651    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:04.983222 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:04.983135    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:05.793579 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:05.793539    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75b21602-f4f7-4b9f-ab4d-4c26548c8b49" path="/var/lib/kubelet/pods/75b21602-f4f7-4b9f-ab4d-4c26548c8b49/volumes"
Sep 08 14:48:06.683022 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:06.682986    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz" podUID="3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" containerName="main" containerID="cri-o://37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de" gracePeriod=45
Sep 08 14:48:06.933495 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:06.933436    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:48:07.017364 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.017334    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tls-certs\") pod \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") "
Sep 08 14:48:07.017364 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.017371    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-dshm\") pod \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") "
Sep 08 14:48:07.017594 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.017394    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tmp-dir\") pod \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") "
Sep 08 14:48:07.017594 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.017455    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-model-cache\") pod \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") "
Sep 08 14:48:07.017594 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.017508    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mknd8\" (UniqueName: \"kubernetes.io/projected/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-kube-api-access-mknd8\") pod \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") "
Sep 08 14:48:07.017594 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.017544    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-home\") pod \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\" (UID: \"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d\") "
Sep 08 14:48:07.017803 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.017721    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" (UID: "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:07.017803 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.017730    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-model-cache" (OuterVolumeSpecName: "model-cache") pod "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" (UID: "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:07.017924 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.017897    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-home" (OuterVolumeSpecName: "home") pod "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" (UID: "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:07.017982 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.017966    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:07.018023 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.017989    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:07.019733 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.019712    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" (UID: "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:48:07.019847 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.019773    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-kube-api-access-mknd8" (OuterVolumeSpecName: "kube-api-access-mknd8") pod "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" (UID: "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d"). InnerVolumeSpecName "kube-api-access-mknd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:48:07.019847 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.019821    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-dshm" (OuterVolumeSpecName: "dshm") pod "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" (UID: "3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:07.119270 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.119227    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mknd8\" (UniqueName: \"kubernetes.io/projected/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-kube-api-access-mknd8\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:07.119270 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.119266    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:07.119270 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.119276    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:07.119270 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.119284    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:07.533549 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.533492    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk" podUID="1163899b-7096-4f22-b7e6-943b85e19b89" containerName="main" containerID="cri-o://28c45065f318148ab7f0fbe8c7260ef4f76d64cdbfc42e1180a4120cc49603f1" gracePeriod=45
Sep 08 14:48:07.712121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.711625    2583 generic.go:356] "Generic (PLEG): container finished" podID="3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" containerID="37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de" exitCode=0
Sep 08 14:48:07.712121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.711719    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"
Sep 08 14:48:07.712121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.711811    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz" event={"ID":"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d","Type":"ContainerDied","Data":"37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de"}
Sep 08 14:48:07.712121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.711867    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz" event={"ID":"3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d","Type":"ContainerDied","Data":"3139b0796e8da2014d6a4a375fe8a19a33b53377c0dc23213058697090e32f47"}
Sep 08 14:48:07.712121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.711889    2583 scope.go:117] "RemoveContainer" containerID="37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de"
Sep 08 14:48:07.715318 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.715291    2583 generic.go:356] "Generic (PLEG): container finished" podID="1163899b-7096-4f22-b7e6-943b85e19b89" containerID="28c45065f318148ab7f0fbe8c7260ef4f76d64cdbfc42e1180a4120cc49603f1" exitCode=0
Sep 08 14:48:07.715450 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.715335    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk" event={"ID":"1163899b-7096-4f22-b7e6-943b85e19b89","Type":"ContainerDied","Data":"28c45065f318148ab7f0fbe8c7260ef4f76d64cdbfc42e1180a4120cc49603f1"}
Sep 08 14:48:07.722573 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.722554    2583 scope.go:117] "RemoveContainer" containerID="37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de"
Sep 08 14:48:07.722886 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:48:07.722865    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de\": container with ID starting with 37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de not found: ID does not exist" containerID="37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de"
Sep 08 14:48:07.722961 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.722894    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de"} err="failed to get container status \"37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de\": rpc error: code = NotFound desc = could not find container \"37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de\": container with ID starting with 37589d73163ce63a755becc80aebed1ccfe35ceebb0a1f77b28342d745b9a7de not found: ID does not exist"
Sep 08 14:48:07.740408 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.740385    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"]
Sep 08 14:48:07.744186 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.744162    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-c8c0e893-kserve-6b7978d8c6-hx2mz"]
Sep 08 14:48:07.794481 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.794406    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" path="/var/lib/kubelet/pods/3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d/volumes"
Sep 08 14:48:07.817907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.817886    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:48:07.927156 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.927126    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-home\") pod \"1163899b-7096-4f22-b7e6-943b85e19b89\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") "
Sep 08 14:48:07.927307 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.927197    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1163899b-7096-4f22-b7e6-943b85e19b89-tls-certs\") pod \"1163899b-7096-4f22-b7e6-943b85e19b89\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") "
Sep 08 14:48:07.927307 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.927250    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdn2b\" (UniqueName: \"kubernetes.io/projected/1163899b-7096-4f22-b7e6-943b85e19b89-kube-api-access-qdn2b\") pod \"1163899b-7096-4f22-b7e6-943b85e19b89\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") "
Sep 08 14:48:07.927307 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.927268    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-dshm\") pod \"1163899b-7096-4f22-b7e6-943b85e19b89\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") "
Sep 08 14:48:07.927307 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.927283    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-model-cache\") pod \"1163899b-7096-4f22-b7e6-943b85e19b89\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") "
Sep 08 14:48:07.927307 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.927297    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-tmp-dir\") pod \"1163899b-7096-4f22-b7e6-943b85e19b89\" (UID: \"1163899b-7096-4f22-b7e6-943b85e19b89\") "
Sep 08 14:48:07.927590 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.927399    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-home" (OuterVolumeSpecName: "home") pod "1163899b-7096-4f22-b7e6-943b85e19b89" (UID: "1163899b-7096-4f22-b7e6-943b85e19b89"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:07.927590 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.927540    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-model-cache" (OuterVolumeSpecName: "model-cache") pod "1163899b-7096-4f22-b7e6-943b85e19b89" (UID: "1163899b-7096-4f22-b7e6-943b85e19b89"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:07.927590 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.927579    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:07.927716 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.927612    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1163899b-7096-4f22-b7e6-943b85e19b89" (UID: "1163899b-7096-4f22-b7e6-943b85e19b89"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:07.929852 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.929807    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1163899b-7096-4f22-b7e6-943b85e19b89-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1163899b-7096-4f22-b7e6-943b85e19b89" (UID: "1163899b-7096-4f22-b7e6-943b85e19b89"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:48:07.929852 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.929843    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1163899b-7096-4f22-b7e6-943b85e19b89-kube-api-access-qdn2b" (OuterVolumeSpecName: "kube-api-access-qdn2b") pod "1163899b-7096-4f22-b7e6-943b85e19b89" (UID: "1163899b-7096-4f22-b7e6-943b85e19b89"). InnerVolumeSpecName "kube-api-access-qdn2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:48:07.930065 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:07.929869    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-dshm" (OuterVolumeSpecName: "dshm") pod "1163899b-7096-4f22-b7e6-943b85e19b89" (UID: "1163899b-7096-4f22-b7e6-943b85e19b89"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:08.028205 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:08.028169    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1163899b-7096-4f22-b7e6-943b85e19b89-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:08.028205 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:08.028204    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdn2b\" (UniqueName: \"kubernetes.io/projected/1163899b-7096-4f22-b7e6-943b85e19b89-kube-api-access-qdn2b\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:08.028602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:08.028221    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:08.028602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:08.028234    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:08.028602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:08.028272    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1163899b-7096-4f22-b7e6-943b85e19b89-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:08.721900 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:08.721868    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"
Sep 08 14:48:08.721900 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:08.721883    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk" event={"ID":"1163899b-7096-4f22-b7e6-943b85e19b89","Type":"ContainerDied","Data":"13944acacfa59b0380ea5c84e4064fba83fb15ee21831bc2a084222d3253e335"}
Sep 08 14:48:08.722167 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:08.721934    2583 scope.go:117] "RemoveContainer" containerID="28c45065f318148ab7f0fbe8c7260ef4f76d64cdbfc42e1180a4120cc49603f1"
Sep 08 14:48:08.769108 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:08.769078    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"]
Sep 08 14:48:08.785358 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:08.785317    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-71518b/autores-model-71518b-kserve-84d776884c-vlwfk"]
Sep 08 14:48:09.615152 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.615109    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9" podUID="a7f5796e-e983-4bd1-b9e0-fb6180ffe219" containerName="main" containerID="cri-o://dd08c6654f141633a78d57932075084037e8b39739b190db48e4b5e6be5d075b" gracePeriod=45
Sep 08 14:48:09.730132 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.730105    2583 generic.go:356] "Generic (PLEG): container finished" podID="a7f5796e-e983-4bd1-b9e0-fb6180ffe219" containerID="dd08c6654f141633a78d57932075084037e8b39739b190db48e4b5e6be5d075b" exitCode=0
Sep 08 14:48:09.730263 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.730175    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9" event={"ID":"a7f5796e-e983-4bd1-b9e0-fb6180ffe219","Type":"ContainerDied","Data":"dd08c6654f141633a78d57932075084037e8b39739b190db48e4b5e6be5d075b"}
Sep 08 14:48:09.796141 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.796113    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1163899b-7096-4f22-b7e6-943b85e19b89" path="/var/lib/kubelet/pods/1163899b-7096-4f22-b7e6-943b85e19b89/volumes"
Sep 08 14:48:09.873937 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.873876    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:48:09.944379 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.944347    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-model-cache\") pod \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") "
Sep 08 14:48:09.944706 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.944409    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-home\") pod \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") "
Sep 08 14:48:09.944706 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.944448    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-dshm\") pod \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") "
Sep 08 14:48:09.944706 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.944475    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2dbl\" (UniqueName: \"kubernetes.io/projected/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-kube-api-access-k2dbl\") pod \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") "
Sep 08 14:48:09.944706 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.944518    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tmp-dir\") pod \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") "
Sep 08 14:48:09.944706 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.944548    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tls-certs\") pod \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\" (UID: \"a7f5796e-e983-4bd1-b9e0-fb6180ffe219\") "
Sep 08 14:48:09.944706 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.944630    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-model-cache" (OuterVolumeSpecName: "model-cache") pod "a7f5796e-e983-4bd1-b9e0-fb6180ffe219" (UID: "a7f5796e-e983-4bd1-b9e0-fb6180ffe219"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:09.945088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.944852    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a7f5796e-e983-4bd1-b9e0-fb6180ffe219" (UID: "a7f5796e-e983-4bd1-b9e0-fb6180ffe219"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:09.945088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.944976    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-home" (OuterVolumeSpecName: "home") pod "a7f5796e-e983-4bd1-b9e0-fb6180ffe219" (UID: "a7f5796e-e983-4bd1-b9e0-fb6180ffe219"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:09.945190 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.945117    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:09.945190 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.945134    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:09.945190 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.945148    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:09.946714 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.946690    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a7f5796e-e983-4bd1-b9e0-fb6180ffe219" (UID: "a7f5796e-e983-4bd1-b9e0-fb6180ffe219"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:48:09.947147 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.947122    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-dshm" (OuterVolumeSpecName: "dshm") pod "a7f5796e-e983-4bd1-b9e0-fb6180ffe219" (UID: "a7f5796e-e983-4bd1-b9e0-fb6180ffe219"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:48:09.947257 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:09.947158    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-kube-api-access-k2dbl" (OuterVolumeSpecName: "kube-api-access-k2dbl") pod "a7f5796e-e983-4bd1-b9e0-fb6180ffe219" (UID: "a7f5796e-e983-4bd1-b9e0-fb6180ffe219"). InnerVolumeSpecName "kube-api-access-k2dbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:48:10.046418 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:10.046388    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:10.046418 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:10.046412    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2dbl\" (UniqueName: \"kubernetes.io/projected/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-kube-api-access-k2dbl\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:10.046418 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:10.046423    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7f5796e-e983-4bd1-b9e0-fb6180ffe219-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:10.613889 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:10.613849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:10.683684 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:10.683648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:10.736944 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:10.736917    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"
Sep 08 14:48:10.736944 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:10.736932    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9" event={"ID":"a7f5796e-e983-4bd1-b9e0-fb6180ffe219","Type":"ContainerDied","Data":"da33ce55242a5c7ff3de7b50a9713289695ae77300f2c94f88cd4b15b6d8d650"}
Sep 08 14:48:10.737150 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:10.736991    2583 scope.go:117] "RemoveContainer" containerID="dd08c6654f141633a78d57932075084037e8b39739b190db48e4b5e6be5d075b"
Sep 08 14:48:10.760659 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:10.760631    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"]
Sep 08 14:48:10.765614 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:10.765594    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-c8c0e893-kserve-8458f567c4-xbdj9"]
Sep 08 14:48:10.864879 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:10.864759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:11.297029 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:11.296996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:11.651992 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:11.651911    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:11.797110 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:11.797073    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7f5796e-e983-4bd1-b9e0-fb6180ffe219" path="/var/lib/kubelet/pods/a7f5796e-e983-4bd1-b9e0-fb6180ffe219/volumes"
Sep 08 14:48:12.081086 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:12.081049    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:14.873966 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:14.873927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:14.973203 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:14.973162    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:16.180938 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.180893    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:16.281950 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.281920    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:16.382312 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.382281    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:16.677136 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.677097    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:16.765253 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.765220    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:16.766275 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766256    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"]
Sep 08 14:48:16.766681 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766667    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7f5796e-e983-4bd1-b9e0-fb6180ffe219" containerName="main"
Sep 08 14:48:16.766742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766683    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7f5796e-e983-4bd1-b9e0-fb6180ffe219" containerName="main"
Sep 08 14:48:16.766742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766699    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" containerName="main"
Sep 08 14:48:16.766742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766705    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" containerName="main"
Sep 08 14:48:16.766742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766714    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1163899b-7096-4f22-b7e6-943b85e19b89" containerName="main"
Sep 08 14:48:16.766742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766724    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1163899b-7096-4f22-b7e6-943b85e19b89" containerName="main"
Sep 08 14:48:16.766742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766739    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4911ea1a-62b6-48f2-bc4d-208cf813bb9b" containerName="payload-processing"
Sep 08 14:48:16.766742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766745    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4911ea1a-62b6-48f2-bc4d-208cf813bb9b" containerName="payload-processing"
Sep 08 14:48:16.767063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766759    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75b21602-f4f7-4b9f-ab4d-4c26548c8b49" containerName="revoke-keys"
Sep 08 14:48:16.767063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766765    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="75b21602-f4f7-4b9f-ab4d-4c26548c8b49" containerName="revoke-keys"
Sep 08 14:48:16.767063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766857    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1163899b-7096-4f22-b7e6-943b85e19b89" containerName="main"
Sep 08 14:48:16.767063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766869    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7f5796e-e983-4bd1-b9e0-fb6180ffe219" containerName="main"
Sep 08 14:48:16.767063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766879    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4911ea1a-62b6-48f2-bc4d-208cf813bb9b" containerName="payload-processing"
Sep 08 14:48:16.767063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766888    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="75b21602-f4f7-4b9f-ab4d-4c26548c8b49" containerName="revoke-keys"
Sep 08 14:48:16.767063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.766896    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cda3e47-ca0c-4e04-a50f-8ea2ac6e238d" containerName="main"
Sep 08 14:48:16.771760 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.771743    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:48:16.774726 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.774703    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-222cd5-dockercfg-wkrv5\""
Sep 08 14:48:16.775217 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.775199    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-222cd5\""
Sep 08 14:48:16.783259 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.783240    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"]
Sep 08 14:48:16.911939 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.911900    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp7dg\" (UniqueName: \"kubernetes.io/projected/655915aa-8767-4fa6-add5-a8d68e8ca550-kube-api-access-qp7dg\") pod \"payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n\" (UID: \"655915aa-8767-4fa6-add5-a8d68e8ca550\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:48:16.912133 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:16.911978    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/655915aa-8767-4fa6-add5-a8d68e8ca550-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n\" (UID: \"655915aa-8767-4fa6-add5-a8d68e8ca550\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:48:17.012461 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:17.012428    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/655915aa-8767-4fa6-add5-a8d68e8ca550-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n\" (UID: \"655915aa-8767-4fa6-add5-a8d68e8ca550\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:48:17.012632 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:17.012519    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp7dg\" (UniqueName: \"kubernetes.io/projected/655915aa-8767-4fa6-add5-a8d68e8ca550-kube-api-access-qp7dg\") pod \"payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n\" (UID: \"655915aa-8767-4fa6-add5-a8d68e8ca550\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:48:17.013194 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:17.013169    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/655915aa-8767-4fa6-add5-a8d68e8ca550-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n\" (UID: \"655915aa-8767-4fa6-add5-a8d68e8ca550\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:48:17.025501 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:17.025476    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp7dg\" (UniqueName: \"kubernetes.io/projected/655915aa-8767-4fa6-add5-a8d68e8ca550-kube-api-access-qp7dg\") pod \"payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n\" (UID: \"655915aa-8767-4fa6-add5-a8d68e8ca550\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:48:17.081997 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:17.081975    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:48:17.355600 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:17.355571    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"]
Sep 08 14:48:17.358327 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:48:17.358301    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod655915aa_8767_4fa6_add5_a8d68e8ca550.slice/crio-72084645597ce87055754fd669c681c082cc4f3860ebd5664e4168c282bf2543 WatchSource:0}: Error finding container 72084645597ce87055754fd669c681c082cc4f3860ebd5664e4168c282bf2543: Status 404 returned error can't find the container with id 72084645597ce87055754fd669c681c082cc4f3860ebd5664e4168c282bf2543
Sep 08 14:48:17.775013 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:17.774976    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n" event={"ID":"655915aa-8767-4fa6-add5-a8d68e8ca550","Type":"ContainerStarted","Data":"95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f"}
Sep 08 14:48:17.775178 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:17.775064    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n" event={"ID":"655915aa-8767-4fa6-add5-a8d68e8ca550","Type":"ContainerStarted","Data":"72084645597ce87055754fd669c681c082cc4f3860ebd5664e4168c282bf2543"}
Sep 08 14:48:17.775250 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:17.775227    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:48:17.794899 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:17.794848    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n" podStartSLOduration=1.794814534 podStartE2EDuration="1.794814534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:48:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:48:17.792656792 +0000 UTC m=+848.520473806" watchObservedRunningTime="2026-09-08 14:48:17.794814534 +0000 UTC m=+848.522631548"
Sep 08 14:48:25.544768 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:25.544739    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:27.982157 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:27.982123    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"]
Sep 08 14:48:27.985793 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:27.985776    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"
Sep 08 14:48:27.988660 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:27.988641    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:48:27.988784 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:27.988683    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 14:48:27.994285 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:27.994262    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"]
Sep 08 14:48:28.109466 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:28.109422    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p\" (UID: \"d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"
Sep 08 14:48:28.210503 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:28.210472    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p\" (UID: \"d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"
Sep 08 14:48:28.211087 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:28.211059    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p\" (UID: \"d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"
Sep 08 14:48:28.296969 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:28.296890    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"
Sep 08 14:48:28.568309 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:28.568285    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"]
Sep 08 14:48:28.570762 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:48:28.570737    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e975f7_7c6c_43dc_a9ed_c8c312bc2a46.slice/crio-55fd26fc58c37e6082e1ff8e9cd72d706db1dc3402ab6dbc8e7bc71409ed32d9 WatchSource:0}: Error finding container 55fd26fc58c37e6082e1ff8e9cd72d706db1dc3402ab6dbc8e7bc71409ed32d9: Status 404 returned error can't find the container with id 55fd26fc58c37e6082e1ff8e9cd72d706db1dc3402ab6dbc8e7bc71409ed32d9
Sep 08 14:48:28.783670 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:28.783647    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:48:28.838958 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:28.838871    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p" event={"ID":"d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46","Type":"ContainerStarted","Data":"ca274711a7600b785e12d135e8780f5fffdd34dfecba84efc60a50cad28d7573"}
Sep 08 14:48:28.839100 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:28.838973    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p" event={"ID":"d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46","Type":"ContainerStarted","Data":"55fd26fc58c37e6082e1ff8e9cd72d706db1dc3402ab6dbc8e7bc71409ed32d9"}
Sep 08 14:48:28.858449 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:28.858399    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p" podStartSLOduration=1.858384102 podStartE2EDuration="1.858384102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:48:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:48:28.856532938 +0000 UTC m=+859.584349953" watchObservedRunningTime="2026-09-08 14:48:28.858384102 +0000 UTC m=+859.586201117"
Sep 08 14:48:29.845459 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:29.845425    2583 generic.go:356] "Generic (PLEG): container finished" podID="d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46" containerID="ca274711a7600b785e12d135e8780f5fffdd34dfecba84efc60a50cad28d7573" exitCode=0
Sep 08 14:48:29.845872 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:29.845514    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p" event={"ID":"d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46","Type":"ContainerDied","Data":"ca274711a7600b785e12d135e8780f5fffdd34dfecba84efc60a50cad28d7573"}
Sep 08 14:48:30.986956 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:30.986932    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"
Sep 08 14:48:31.135673 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:31.135583    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46-maas-api-service-ca\") pod \"d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46\" (UID: \"d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46\") "
Sep 08 14:48:31.135968 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:31.135944    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46" (UID: "d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:48:31.237134 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:31.237103    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:48:31.857172 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:31.857143    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"
Sep 08 14:48:31.857320 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:31.857175    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p" event={"ID":"d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46","Type":"ContainerDied","Data":"55fd26fc58c37e6082e1ff8e9cd72d706db1dc3402ab6dbc8e7bc71409ed32d9"}
Sep 08 14:48:31.857320 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:31.857216    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55fd26fc58c37e6082e1ff8e9cd72d706db1dc3402ab6dbc8e7bc71409ed32d9"
Sep 08 14:48:35.082143 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:35.082111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:35.159275 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:35.159241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:35.230753 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:35.230712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:35.290713 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:35.290679    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"]
Sep 08 14:48:35.294355 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:35.294334    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-02cfd933-tcb7p"]
Sep 08 14:48:35.724789 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:35.724760    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:35.793679 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:35.793651    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46" path="/var/lib/kubelet/pods/d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46/volumes"
Sep 08 14:48:35.819590 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:35.819558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:36.132062 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:36.131976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:36.223785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:36.223752    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:36.327892 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:36.327857    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:37.065560 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:37.065533    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:37.636383 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:37.636348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:39.792948 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:39.792902    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:43.760750 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:43.760720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:43.877117 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:43.877079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:43.976011 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:43.975974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:44.080094 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:44.080012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:44.458453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:44.458375    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:48.408699 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:48.408667    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:53.470776 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.470743    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"]
Sep 08 14:48:53.471186 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.471172    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46" containerName="revoke-keys"
Sep 08 14:48:53.471232 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.471189    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46" containerName="revoke-keys"
Sep 08 14:48:53.471286 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.471277    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5e975f7-7c6c-43dc-a9ed-c8c312bc2a46" containerName="revoke-keys"
Sep 08 14:48:53.474630 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.474613    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.477594 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.477573    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-02a199\"/\"openshift-service-ca.crt\""
Sep 08 14:48:53.478823 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.478800    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-02a199\"/\"kube-root-ca.crt\""
Sep 08 14:48:53.478948 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.478804    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-02a199\"/\"test-model-02a199-kserve-self-signed-certs\""
Sep 08 14:48:53.478996 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.478971    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-02a199\"/\"default-dockercfg-8zclq\""
Sep 08 14:48:53.484712 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.484692    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"]
Sep 08 14:48:53.503010 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.502977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:53.533008 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.532985    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-home\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.533126 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.533017    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-dshm\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.533126 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.533089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-tmp-dir\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.533126 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.533106    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55f542a3-ff20-488a-87f3-29219eb39473-tls-certs\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.533240 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.533148    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-model-cache\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.533240 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.533196    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grxpc\" (UniqueName: \"kubernetes.io/projected/55f542a3-ff20-488a-87f3-29219eb39473-kube-api-access-grxpc\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.634144 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.634111    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-tmp-dir\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.634144 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.634146    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55f542a3-ff20-488a-87f3-29219eb39473-tls-certs\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.634382 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.634166    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-model-cache\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.634382 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.634190    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grxpc\" (UniqueName: \"kubernetes.io/projected/55f542a3-ff20-488a-87f3-29219eb39473-kube-api-access-grxpc\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.634382 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.634239    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-home\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.634382 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.634259    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-dshm\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.634601 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.634573    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-tmp-dir\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.634662 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.634615    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-model-cache\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.634702 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.634660    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-home\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.636602 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.636582    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-dshm\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.636709 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.636693    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55f542a3-ff20-488a-87f3-29219eb39473-tls-certs\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.643243 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.643219    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grxpc\" (UniqueName: \"kubernetes.io/projected/55f542a3-ff20-488a-87f3-29219eb39473-kube-api-access-grxpc\") pod \"test-model-02a199-kserve-f4d8dd885-b88v7\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") " pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:53.806771 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:53.806687    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:48:54.086045 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:54.086005    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"]
Sep 08 14:48:54.088188 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:48:54.088163    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55f542a3_ff20_488a_87f3_29219eb39473.slice/crio-fd0b4fa63878e8659500b6be13aadcfccebbec56ac03922684335174088d0f50 WatchSource:0}: Error finding container fd0b4fa63878e8659500b6be13aadcfccebbec56ac03922684335174088d0f50: Status 404 returned error can't find the container with id fd0b4fa63878e8659500b6be13aadcfccebbec56ac03922684335174088d0f50
Sep 08 14:48:54.971326 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:54.971288    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7" event={"ID":"55f542a3-ff20-488a-87f3-29219eb39473","Type":"ContainerStarted","Data":"7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff"}
Sep 08 14:48:54.971326 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:54.971331    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7" event={"ID":"55f542a3-ff20-488a-87f3-29219eb39473","Type":"ContainerStarted","Data":"fd0b4fa63878e8659500b6be13aadcfccebbec56ac03922684335174088d0f50"}
Sep 08 14:48:54.992597 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:54.992550    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7" podStartSLOduration=1.99253492 podStartE2EDuration="1.99253492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:48:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:48:54.991739393 +0000 UTC m=+885.719556407" watchObservedRunningTime="2026-09-08 14:48:54.99253492 +0000 UTC m=+885.720351933"
Sep 08 14:48:58.978114 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:58.978077    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"]
Sep 08 14:48:58.981674 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:58.981656    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:58.987119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:58.987097    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-222cd5\"/\"kube-root-ca.crt\""
Sep 08 14:48:58.987244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:58.987143    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-222cd5\"/\"openshift-service-ca.crt\""
Sep 08 14:48:58.987244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:58.987103    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-222cd5\"/\"explicit-model-222cd5-kserve-self-signed-certs\""
Sep 08 14:48:58.987244 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:58.987099    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-222cd5\"/\"default-dockercfg-fw4rw\""
Sep 08 14:48:58.997778 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:58.997752    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"]
Sep 08 14:48:59.084367 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.084318    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:48:59.085437 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.085420    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-model-cache\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.085510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.085465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-home\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.085510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.085482    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tmp-dir\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.085510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.085499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nfmw\" (UniqueName: \"kubernetes.io/projected/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-kube-api-access-7nfmw\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.085646 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.085559    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-dshm\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.085646 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.085598    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tls-certs\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.186494 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.186458    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-home\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.186494 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.186493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tmp-dir\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.186708 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.186510    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nfmw\" (UniqueName: \"kubernetes.io/projected/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-kube-api-access-7nfmw\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.186708 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.186628    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-dshm\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.186860 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.186786    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tls-certs\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.186938 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.186918    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-model-cache\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.186987 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.186970    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tmp-dir\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.187040 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.186921    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-home\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.187135 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.187117    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-model-cache\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.189018 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.188996    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-dshm\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.189256 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.189237    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tls-certs\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.197003 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.196976    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nfmw\" (UniqueName: \"kubernetes.io/projected/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-kube-api-access-7nfmw\") pod \"explicit-model-222cd5-kserve-6c5f655689-kf4qb\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") " pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.291782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.291712    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:48:59.567327 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:48:59.567301    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"]
Sep 08 14:48:59.569140 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:48:59.569108    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4eaa39a_ba74_4d22_b179_37acaadb6b9f.slice/crio-3b2a26ad0c24c843f2717067e8166ae0583beb1802dd9ed1460bd2f1d6c565d6 WatchSource:0}: Error finding container 3b2a26ad0c24c843f2717067e8166ae0583beb1802dd9ed1460bd2f1d6c565d6: Status 404 returned error can't find the container with id 3b2a26ad0c24c843f2717067e8166ae0583beb1802dd9ed1460bd2f1d6c565d6
Sep 08 14:49:00.002700 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:00.002656    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb" event={"ID":"b4eaa39a-ba74-4d22-b179-37acaadb6b9f","Type":"ContainerStarted","Data":"5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6"}
Sep 08 14:49:00.002700 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:00.002699    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb" event={"ID":"b4eaa39a-ba74-4d22-b179-37acaadb6b9f","Type":"ContainerStarted","Data":"3b2a26ad0c24c843f2717067e8166ae0583beb1802dd9ed1460bd2f1d6c565d6"}
Sep 08 14:49:00.021769 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:00.021724    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb" podStartSLOduration=2.021708493 podStartE2EDuration="2.021708493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:48:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:49:00.020043613 +0000 UTC m=+890.747860628" watchObservedRunningTime="2026-09-08 14:49:00.021708493 +0000 UTC m=+890.749525508"
Sep 08 14:49:03.807742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:03.807710    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:49:03.808232 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:03.807758    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:49:03.820554 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:03.820530    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:49:04.034295 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:04.034262    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:49:09.039756 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:09.039719    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"]
Sep 08 14:49:09.070027 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:09.069981    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:09.292639 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:09.292569    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:49:09.330415 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:09.330386    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:10.461228 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:10.461189    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:16.576136 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:16.576098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:16.687763 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:16.687734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:16.799967 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:16.799930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:18.286281 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:18.286247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:19.673866 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:19.672073    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"]
Sep 08 14:49:19.673866 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:19.673288    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n" podUID="655915aa-8767-4fa6-add5-a8d68e8ca550" containerName="payload-pre-processing" containerID="cri-o://95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f" gracePeriod=30
Sep 08 14:49:19.774090 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:19.774050    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:19.851048 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:19.851010    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:19.932854 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:19.932776    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:49:20.077737 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.077702    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/655915aa-8767-4fa6-add5-a8d68e8ca550-plugins-config-volume\") pod \"655915aa-8767-4fa6-add5-a8d68e8ca550\" (UID: \"655915aa-8767-4fa6-add5-a8d68e8ca550\") "
Sep 08 14:49:20.077935 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.077800    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp7dg\" (UniqueName: \"kubernetes.io/projected/655915aa-8767-4fa6-add5-a8d68e8ca550-kube-api-access-qp7dg\") pod \"655915aa-8767-4fa6-add5-a8d68e8ca550\" (UID: \"655915aa-8767-4fa6-add5-a8d68e8ca550\") "
Sep 08 14:49:20.078139 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.078114    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/655915aa-8767-4fa6-add5-a8d68e8ca550-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "655915aa-8767-4fa6-add5-a8d68e8ca550" (UID: "655915aa-8767-4fa6-add5-a8d68e8ca550"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:49:20.079942 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.079907    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/655915aa-8767-4fa6-add5-a8d68e8ca550-kube-api-access-qp7dg" (OuterVolumeSpecName: "kube-api-access-qp7dg") pod "655915aa-8767-4fa6-add5-a8d68e8ca550" (UID: "655915aa-8767-4fa6-add5-a8d68e8ca550"). InnerVolumeSpecName "kube-api-access-qp7dg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:49:20.106766 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.106733    2583 generic.go:356] "Generic (PLEG): container finished" podID="655915aa-8767-4fa6-add5-a8d68e8ca550" containerID="95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f" exitCode=0
Sep 08 14:49:20.106897 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.106773    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"
Sep 08 14:49:20.106897 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.106841    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n" event={"ID":"655915aa-8767-4fa6-add5-a8d68e8ca550","Type":"ContainerDied","Data":"95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f"}
Sep 08 14:49:20.106897 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.106892    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n" event={"ID":"655915aa-8767-4fa6-add5-a8d68e8ca550","Type":"ContainerDied","Data":"72084645597ce87055754fd669c681c082cc4f3860ebd5664e4168c282bf2543"}
Sep 08 14:49:20.107017 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.106914    2583 scope.go:117] "RemoveContainer" containerID="95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f"
Sep 08 14:49:20.116021 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.116009    2583 scope.go:117] "RemoveContainer" containerID="95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f"
Sep 08 14:49:20.116273 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:49:20.116258    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f\": container with ID starting with 95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f not found: ID does not exist" containerID="95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f"
Sep 08 14:49:20.116312 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.116279    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f"} err="failed to get container status \"95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f\": rpc error: code = NotFound desc = could not find container \"95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f\": container with ID starting with 95b51ee4c861d2fcec2b1731d885a1e81d3ffeaf4b9b3c2568f490a2926d496f not found: ID does not exist"
Sep 08 14:49:20.129685 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.129663    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"]
Sep 08 14:49:20.134056 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.134034    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-222cd5-65db8c7cdb-7pp7n"]
Sep 08 14:49:20.179106 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.179080    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp7dg\" (UniqueName: \"kubernetes.io/projected/655915aa-8767-4fa6-add5-a8d68e8ca550-kube-api-access-qp7dg\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:49:20.179106 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.179104    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/655915aa-8767-4fa6-add5-a8d68e8ca550-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:49:20.201126 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:20.201067    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:21.793939 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:21.793910    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="655915aa-8767-4fa6-add5-a8d68e8ca550" path="/var/lib/kubelet/pods/655915aa-8767-4fa6-add5-a8d68e8ca550/volumes"
Sep 08 14:49:24.063236 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.063193    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb" podUID="b4eaa39a-ba74-4d22-b179-37acaadb6b9f" containerName="main" containerID="cri-o://5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6" gracePeriod=45
Sep 08 14:49:24.319448 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.319393    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:49:24.415382 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.415346    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tls-certs\") pod \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") "
Sep 08 14:49:24.415561 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.415400    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tmp-dir\") pod \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") "
Sep 08 14:49:24.415561 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.415444    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-model-cache\") pod \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") "
Sep 08 14:49:24.415689 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.415581    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-dshm\") pod \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") "
Sep 08 14:49:24.415689 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.415648    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-model-cache" (OuterVolumeSpecName: "model-cache") pod "b4eaa39a-ba74-4d22-b179-37acaadb6b9f" (UID: "b4eaa39a-ba74-4d22-b179-37acaadb6b9f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:49:24.415689 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.415659    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nfmw\" (UniqueName: \"kubernetes.io/projected/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-kube-api-access-7nfmw\") pod \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") "
Sep 08 14:49:24.415689 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.415675    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b4eaa39a-ba74-4d22-b179-37acaadb6b9f" (UID: "b4eaa39a-ba74-4d22-b179-37acaadb6b9f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:49:24.415689 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.415687    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-home\") pod \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\" (UID: \"b4eaa39a-ba74-4d22-b179-37acaadb6b9f\") "
Sep 08 14:49:24.416024 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.415999    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-home" (OuterVolumeSpecName: "home") pod "b4eaa39a-ba74-4d22-b179-37acaadb6b9f" (UID: "b4eaa39a-ba74-4d22-b179-37acaadb6b9f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:49:24.416146 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.416044    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:49:24.416146 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.416063    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:49:24.417895 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.417876    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b4eaa39a-ba74-4d22-b179-37acaadb6b9f" (UID: "b4eaa39a-ba74-4d22-b179-37acaadb6b9f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:49:24.417976 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.417895    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-dshm" (OuterVolumeSpecName: "dshm") pod "b4eaa39a-ba74-4d22-b179-37acaadb6b9f" (UID: "b4eaa39a-ba74-4d22-b179-37acaadb6b9f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:49:24.417976 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.417966    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-kube-api-access-7nfmw" (OuterVolumeSpecName: "kube-api-access-7nfmw") pod "b4eaa39a-ba74-4d22-b179-37acaadb6b9f" (UID: "b4eaa39a-ba74-4d22-b179-37acaadb6b9f"). InnerVolumeSpecName "kube-api-access-7nfmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:49:24.516531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.516502    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:49:24.516531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.516526    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:49:24.516531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.516536    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nfmw\" (UniqueName: \"kubernetes.io/projected/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-kube-api-access-7nfmw\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:49:24.516732 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:24.516546    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4eaa39a-ba74-4d22-b179-37acaadb6b9f-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:49:25.133881 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:25.133848    2583 generic.go:356] "Generic (PLEG): container finished" podID="b4eaa39a-ba74-4d22-b179-37acaadb6b9f" containerID="5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6" exitCode=0
Sep 08 14:49:25.134302 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:25.133888    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"
Sep 08 14:49:25.134302 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:25.133902    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb" event={"ID":"b4eaa39a-ba74-4d22-b179-37acaadb6b9f","Type":"ContainerDied","Data":"5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6"}
Sep 08 14:49:25.134302 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:25.133940    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb" event={"ID":"b4eaa39a-ba74-4d22-b179-37acaadb6b9f","Type":"ContainerDied","Data":"3b2a26ad0c24c843f2717067e8166ae0583beb1802dd9ed1460bd2f1d6c565d6"}
Sep 08 14:49:25.134302 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:25.133956    2583 scope.go:117] "RemoveContainer" containerID="5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6"
Sep 08 14:49:25.145973 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:25.145950    2583 scope.go:117] "RemoveContainer" containerID="5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6"
Sep 08 14:49:25.146455 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:49:25.146429    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6\": container with ID starting with 5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6 not found: ID does not exist" containerID="5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6"
Sep 08 14:49:25.146539 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:25.146467    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6"} err="failed to get container status \"5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6\": rpc error: code = NotFound desc = could not find container \"5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6\": container with ID starting with 5b955cbe9ded015632ef90a84e61f520a42fd20c60874d3095c627102e4eeca6 not found: ID does not exist"
Sep 08 14:49:25.157117 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:25.157093    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"]
Sep 08 14:49:25.161210 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:25.161190    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-222cd5/explicit-model-222cd5-kserve-6c5f655689-kf4qb"]
Sep 08 14:49:25.793319 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:25.793283    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4eaa39a-ba74-4d22-b179-37acaadb6b9f" path="/var/lib/kubelet/pods/b4eaa39a-ba74-4d22-b179-37acaadb6b9f/volumes"
Sep 08 14:49:26.532463 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:26.532429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:26.602758 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:26.602726    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:26.809329 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:26.809254    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:27.248624 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:27.248590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:27.663814 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:27.663734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:28.126628 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:28.126586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:29.228238 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:29.228194    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:31.100810 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:31.100779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:31.201582 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:31.201547    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:32.614113 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:32.614080    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:38.828499 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:38.828465    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:42.014140 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:42.014105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:43.773695 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.773667    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"]
Sep 08 14:49:43.774094 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.774083    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4eaa39a-ba74-4d22-b179-37acaadb6b9f" containerName="main"
Sep 08 14:49:43.774136 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.774096    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4eaa39a-ba74-4d22-b179-37acaadb6b9f" containerName="main"
Sep 08 14:49:43.774136 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.774104    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="655915aa-8767-4fa6-add5-a8d68e8ca550" containerName="payload-pre-processing"
Sep 08 14:49:43.774136 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.774110    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="655915aa-8767-4fa6-add5-a8d68e8ca550" containerName="payload-pre-processing"
Sep 08 14:49:43.774230 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.774192    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="655915aa-8767-4fa6-add5-a8d68e8ca550" containerName="payload-pre-processing"
Sep 08 14:49:43.774230 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.774204    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4eaa39a-ba74-4d22-b179-37acaadb6b9f" containerName="main"
Sep 08 14:49:43.777242 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.777223    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.782077 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.782049    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-f020ee\"/\"kube-root-ca.crt\""
Sep 08 14:49:43.782077 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.782066    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-f020ee\"/\"noten-model-f020ee-kserve-self-signed-certs\""
Sep 08 14:49:43.782264 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.782138    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-f020ee\"/\"openshift-service-ca.crt\""
Sep 08 14:49:43.782264 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.782200    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-f020ee\"/\"default-dockercfg-nhszs\""
Sep 08 14:49:43.798383 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.798349    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"]
Sep 08 14:49:43.863915 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.863883    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:43.882284 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.882260    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-dshm\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.882407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.882285    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecf62e9-667e-4a12-963e-72f7922639db-tls-certs\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.882407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.882332    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dds5l\" (UniqueName: \"kubernetes.io/projected/5ecf62e9-667e-4a12-963e-72f7922639db-kube-api-access-dds5l\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.882407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.882351    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-home\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.882407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.882390    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-model-cache\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.882599 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.882530    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-tmp-dir\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.983175 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.983142    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-dshm\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.983175 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.983173    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecf62e9-667e-4a12-963e-72f7922639db-tls-certs\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.983423 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.983208    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dds5l\" (UniqueName: \"kubernetes.io/projected/5ecf62e9-667e-4a12-963e-72f7922639db-kube-api-access-dds5l\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.983423 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.983305    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-home\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.983423 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.983372    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-model-cache\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.983423 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.983418    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-tmp-dir\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.983679 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.983659    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-home\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.983760 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.983733    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-model-cache\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.983816 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.983769    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-tmp-dir\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.985544 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.985528    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-dshm\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.985919 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.985902    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecf62e9-667e-4a12-963e-72f7922639db-tls-certs\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:43.992988 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:43.992966    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dds5l\" (UniqueName: \"kubernetes.io/projected/5ecf62e9-667e-4a12-963e-72f7922639db-kube-api-access-dds5l\") pod \"noten-model-f020ee-kserve-7c5c7c6475-2n656\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") " pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:44.086529 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:44.086450    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:44.360493 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:44.360469    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"]
Sep 08 14:49:44.362796 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:49:44.362764    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ecf62e9_667e_4a12_963e_72f7922639db.slice/crio-66f2e75826a5210c49fba43f967409b599cd140ffbe67dcd2f8fd75bab810cf8 WatchSource:0}: Error finding container 66f2e75826a5210c49fba43f967409b599cd140ffbe67dcd2f8fd75bab810cf8: Status 404 returned error can't find the container with id 66f2e75826a5210c49fba43f967409b599cd140ffbe67dcd2f8fd75bab810cf8
Sep 08 14:49:45.237340 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:45.237306    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656" event={"ID":"5ecf62e9-667e-4a12-963e-72f7922639db","Type":"ContainerStarted","Data":"45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d"}
Sep 08 14:49:45.237740 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:45.237349    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656" event={"ID":"5ecf62e9-667e-4a12-963e-72f7922639db","Type":"ContainerStarted","Data":"66f2e75826a5210c49fba43f967409b599cd140ffbe67dcd2f8fd75bab810cf8"}
Sep 08 14:49:45.255883 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:45.255814    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656" podStartSLOduration=2.255799587 podStartE2EDuration="2.255799587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:49:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:49:45.255790658 +0000 UTC m=+935.983607672" watchObservedRunningTime="2026-09-08 14:49:45.255799587 +0000 UTC m=+935.983616684"
Sep 08 14:49:49.542204 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:49.542170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:50.290490 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:50.290444    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:54.087534 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:54.087490    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:54.087534 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:54.087541    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:54.100094 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:54.100072    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:54.291414 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:54.291383    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:49:56.166330 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:56.166294    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:57.836263 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:57.836230    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:57.912075 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:57.912043    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:58.779434 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:58.779395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:58.921473 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:58.921445    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"]
Sep 08 14:49:58.945146 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:58.945108    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:59.047655 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:59.047575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:59.250594 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:59.250564    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:49:59.366100 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:49:59.366021    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:00.831790 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:00.831752    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:01.801190 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:01.801157    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"]
Sep 08 14:50:01.913847 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:01.913802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:02.041033 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:02.040984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:04.532822 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:04.532780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:05.585473 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:05.585439    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:05.653190 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:05.653159    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:06.151041 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:06.151005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:09.961996 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:09.961963    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:12.219004 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.218952    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:12.301252 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.301222    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:12.436319 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.436279    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:12.453630 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.453595    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"]
Sep 08 14:50:12.453886 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.453858    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw" podUID="d169e6eb-618c-4fb6-b214-164a0a6ffb68" containerName="payload-processing" containerID="cri-o://bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40" gracePeriod=30
Sep 08 14:50:12.461686 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.461365    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"]
Sep 08 14:50:12.461686 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.461589    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr" podUID="64c2b1e9-e3cb-4a65-b016-5e59d1cef488" containerName="payload-pre-processing" containerID="cri-o://a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826" gracePeriod=30
Sep 08 14:50:12.550469 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.550442    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:12.657071 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.657039    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:12.745173 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.745149    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:50:12.747116 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.747095    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:50:12.783696 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.783669    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:12.841115 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.841091    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2g2k6\" (UniqueName: \"kubernetes.io/projected/d169e6eb-618c-4fb6-b214-164a0a6ffb68-kube-api-access-2g2k6\") pod \"d169e6eb-618c-4fb6-b214-164a0a6ffb68\" (UID: \"d169e6eb-618c-4fb6-b214-164a0a6ffb68\") "
Sep 08 14:50:12.841272 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.841165    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d169e6eb-618c-4fb6-b214-164a0a6ffb68-plugins-config-volume\") pod \"d169e6eb-618c-4fb6-b214-164a0a6ffb68\" (UID: \"d169e6eb-618c-4fb6-b214-164a0a6ffb68\") "
Sep 08 14:50:12.841555 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.841530    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d169e6eb-618c-4fb6-b214-164a0a6ffb68-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d169e6eb-618c-4fb6-b214-164a0a6ffb68" (UID: "d169e6eb-618c-4fb6-b214-164a0a6ffb68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:50:12.843417 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.843392    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d169e6eb-618c-4fb6-b214-164a0a6ffb68-kube-api-access-2g2k6" (OuterVolumeSpecName: "kube-api-access-2g2k6") pod "d169e6eb-618c-4fb6-b214-164a0a6ffb68" (UID: "d169e6eb-618c-4fb6-b214-164a0a6ffb68"). InnerVolumeSpecName "kube-api-access-2g2k6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:50:12.874955 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.874929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:12.942047 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.942008    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-plugins-config-volume\") pod \"64c2b1e9-e3cb-4a65-b016-5e59d1cef488\" (UID: \"64c2b1e9-e3cb-4a65-b016-5e59d1cef488\") "
Sep 08 14:50:12.942165 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.942114    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp8zq\" (UniqueName: \"kubernetes.io/projected/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-kube-api-access-dp8zq\") pod \"64c2b1e9-e3cb-4a65-b016-5e59d1cef488\" (UID: \"64c2b1e9-e3cb-4a65-b016-5e59d1cef488\") "
Sep 08 14:50:12.942375 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.942341    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64c2b1e9-e3cb-4a65-b016-5e59d1cef488" (UID: "64c2b1e9-e3cb-4a65-b016-5e59d1cef488"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:50:12.942683 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.942660    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d169e6eb-618c-4fb6-b214-164a0a6ffb68-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:12.942771 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.942688    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2g2k6\" (UniqueName: \"kubernetes.io/projected/d169e6eb-618c-4fb6-b214-164a0a6ffb68-kube-api-access-2g2k6\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:12.942771 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.942708    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:12.944643 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:12.944623    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-kube-api-access-dp8zq" (OuterVolumeSpecName: "kube-api-access-dp8zq") pod "64c2b1e9-e3cb-4a65-b016-5e59d1cef488" (UID: "64c2b1e9-e3cb-4a65-b016-5e59d1cef488"). InnerVolumeSpecName "kube-api-access-dp8zq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:50:13.043340 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.043267    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp8zq\" (UniqueName: \"kubernetes.io/projected/64c2b1e9-e3cb-4a65-b016-5e59d1cef488-kube-api-access-dp8zq\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:13.109400 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.109362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:13.380106 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.380015    2583 generic.go:356] "Generic (PLEG): container finished" podID="d169e6eb-618c-4fb6-b214-164a0a6ffb68" containerID="bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40" exitCode=0
Sep 08 14:50:13.380106 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.380056    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"
Sep 08 14:50:13.380106 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.380098    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw" event={"ID":"d169e6eb-618c-4fb6-b214-164a0a6ffb68","Type":"ContainerDied","Data":"bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40"}
Sep 08 14:50:13.380634 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.380144    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw" event={"ID":"d169e6eb-618c-4fb6-b214-164a0a6ffb68","Type":"ContainerDied","Data":"0774ffb46664cad5ceb108165a7a7bc28b9955a4464b6c6a01d97bdaf573d306"}
Sep 08 14:50:13.380634 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.380174    2583 scope.go:117] "RemoveContainer" containerID="bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40"
Sep 08 14:50:13.381621 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.381598    2583 generic.go:356] "Generic (PLEG): container finished" podID="64c2b1e9-e3cb-4a65-b016-5e59d1cef488" containerID="a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826" exitCode=0
Sep 08 14:50:13.381740 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.381640    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"
Sep 08 14:50:13.381740 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.381654    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr" event={"ID":"64c2b1e9-e3cb-4a65-b016-5e59d1cef488","Type":"ContainerDied","Data":"a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826"}
Sep 08 14:50:13.381740 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.381685    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr" event={"ID":"64c2b1e9-e3cb-4a65-b016-5e59d1cef488","Type":"ContainerDied","Data":"9f29bd1a5ae90236d34283fa3951bbb1e5d6506ed6fcbb744224ad86a68f4b97"}
Sep 08 14:50:13.390728 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.390709    2583 scope.go:117] "RemoveContainer" containerID="bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40"
Sep 08 14:50:13.391096 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:50:13.391073    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40\": container with ID starting with bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40 not found: ID does not exist" containerID="bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40"
Sep 08 14:50:13.391188 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.391106    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40"} err="failed to get container status \"bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40\": rpc error: code = NotFound desc = could not find container \"bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40\": container with ID starting with bfde5d03f7dbf76e36d9ef67b69a52c7c3773c1ac3af8539f7ca3a304d927c40 not found: ID does not exist"
Sep 08 14:50:13.391188 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.391129    2583 scope.go:117] "RemoveContainer" containerID="a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826"
Sep 08 14:50:13.399009 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.398995    2583 scope.go:117] "RemoveContainer" containerID="a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826"
Sep 08 14:50:13.399250 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:50:13.399230    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826\": container with ID starting with a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826 not found: ID does not exist" containerID="a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826"
Sep 08 14:50:13.399303 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.399255    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826"} err="failed to get container status \"a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826\": rpc error: code = NotFound desc = could not find container \"a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826\": container with ID starting with a2eb6806cd5dd9fbe461078d48c6fca868ad84b5c6bfce1f8eeba19741898826 not found: ID does not exist"
Sep 08 14:50:13.406757 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.406733    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"]
Sep 08 14:50:13.411938 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.411917    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-798a5d-84f79d4cc4-xcdwr"]
Sep 08 14:50:13.423553 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.423522    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"]
Sep 08 14:50:13.427366 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.427347    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-798a5d-7748cb9885-rrdkw"]
Sep 08 14:50:13.793983 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.793952    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64c2b1e9-e3cb-4a65-b016-5e59d1cef488" path="/var/lib/kubelet/pods/64c2b1e9-e3cb-4a65-b016-5e59d1cef488/volumes"
Sep 08 14:50:13.794327 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.794314    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d169e6eb-618c-4fb6-b214-164a0a6ffb68" path="/var/lib/kubelet/pods/d169e6eb-618c-4fb6-b214-164a0a6ffb68/volumes"
Sep 08 14:50:13.941804 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:13.941769    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656" podUID="5ecf62e9-667e-4a12-963e-72f7922639db" containerName="main" containerID="cri-o://45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d" gracePeriod=45
Sep 08 14:50:14.190971 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.190942    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:50:14.355406 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.355322    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecf62e9-667e-4a12-963e-72f7922639db-tls-certs\") pod \"5ecf62e9-667e-4a12-963e-72f7922639db\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") "
Sep 08 14:50:14.355406 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.355362    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-tmp-dir\") pod \"5ecf62e9-667e-4a12-963e-72f7922639db\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") "
Sep 08 14:50:14.355406 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.355407    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dds5l\" (UniqueName: \"kubernetes.io/projected/5ecf62e9-667e-4a12-963e-72f7922639db-kube-api-access-dds5l\") pod \"5ecf62e9-667e-4a12-963e-72f7922639db\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") "
Sep 08 14:50:14.355704 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.355439    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-model-cache\") pod \"5ecf62e9-667e-4a12-963e-72f7922639db\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") "
Sep 08 14:50:14.355704 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.355455    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-home\") pod \"5ecf62e9-667e-4a12-963e-72f7922639db\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") "
Sep 08 14:50:14.355704 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.355491    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-dshm\") pod \"5ecf62e9-667e-4a12-963e-72f7922639db\" (UID: \"5ecf62e9-667e-4a12-963e-72f7922639db\") "
Sep 08 14:50:14.355704 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.355654    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5ecf62e9-667e-4a12-963e-72f7922639db" (UID: "5ecf62e9-667e-4a12-963e-72f7922639db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:50:14.355928 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.355706    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-model-cache" (OuterVolumeSpecName: "model-cache") pod "5ecf62e9-667e-4a12-963e-72f7922639db" (UID: "5ecf62e9-667e-4a12-963e-72f7922639db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:50:14.355928 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.355746    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-home" (OuterVolumeSpecName: "home") pod "5ecf62e9-667e-4a12-963e-72f7922639db" (UID: "5ecf62e9-667e-4a12-963e-72f7922639db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:50:14.355928 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.355770    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:14.355928 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.355787    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:14.357769 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.357738    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ecf62e9-667e-4a12-963e-72f7922639db-kube-api-access-dds5l" (OuterVolumeSpecName: "kube-api-access-dds5l") pod "5ecf62e9-667e-4a12-963e-72f7922639db" (UID: "5ecf62e9-667e-4a12-963e-72f7922639db"). InnerVolumeSpecName "kube-api-access-dds5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:50:14.357962 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.357775    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ecf62e9-667e-4a12-963e-72f7922639db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5ecf62e9-667e-4a12-963e-72f7922639db" (UID: "5ecf62e9-667e-4a12-963e-72f7922639db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:50:14.357962 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.357789    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-dshm" (OuterVolumeSpecName: "dshm") pod "5ecf62e9-667e-4a12-963e-72f7922639db" (UID: "5ecf62e9-667e-4a12-963e-72f7922639db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:50:14.389897 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.389869    2583 generic.go:356] "Generic (PLEG): container finished" podID="5ecf62e9-667e-4a12-963e-72f7922639db" containerID="45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d" exitCode=0
Sep 08 14:50:14.390227 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.389902    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656" event={"ID":"5ecf62e9-667e-4a12-963e-72f7922639db","Type":"ContainerDied","Data":"45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d"}
Sep 08 14:50:14.390227 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.389929    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"
Sep 08 14:50:14.390227 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.389946    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656" event={"ID":"5ecf62e9-667e-4a12-963e-72f7922639db","Type":"ContainerDied","Data":"66f2e75826a5210c49fba43f967409b599cd140ffbe67dcd2f8fd75bab810cf8"}
Sep 08 14:50:14.390227 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.389971    2583 scope.go:117] "RemoveContainer" containerID="45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d"
Sep 08 14:50:14.398619 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.398605    2583 scope.go:117] "RemoveContainer" containerID="45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d"
Sep 08 14:50:14.398922 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:50:14.398897    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d\": container with ID starting with 45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d not found: ID does not exist" containerID="45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d"
Sep 08 14:50:14.399021 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.398932    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d"} err="failed to get container status \"45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d\": rpc error: code = NotFound desc = could not find container \"45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d\": container with ID starting with 45d53456b7aefd6621bbb7666149a2b8c3874d9397dede2a3e087dd7a0d2223d not found: ID does not exist"
Sep 08 14:50:14.412549 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.412525    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"]
Sep 08 14:50:14.416057 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.416038    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-f020ee/noten-model-f020ee-kserve-7c5c7c6475-2n656"]
Sep 08 14:50:14.456679 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.456653    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:14.456679 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.456676    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ecf62e9-667e-4a12-963e-72f7922639db-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:14.456812 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.456685    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dds5l\" (UniqueName: \"kubernetes.io/projected/5ecf62e9-667e-4a12-963e-72f7922639db-kube-api-access-dds5l\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:14.456812 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:14.456695    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ecf62e9-667e-4a12-963e-72f7922639db-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:15.570158 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:15.570116    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:15.671146 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:15.671117    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:15.794845 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:15.794795    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ecf62e9-667e-4a12-963e-72f7922639db" path="/var/lib/kubelet/pods/5ecf62e9-667e-4a12-963e-72f7922639db/volumes"
Sep 08 14:50:16.824018 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:16.823975    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7" podUID="55f542a3-ff20-488a-87f3-29219eb39473" containerName="main" containerID="cri-o://7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff" gracePeriod=45
Sep 08 14:50:16.832376 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:16.832350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:17.071952 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.071929    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:50:17.077179 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077138    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-model-cache\") pod \"55f542a3-ff20-488a-87f3-29219eb39473\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") "
Sep 08 14:50:17.077239 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077195    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-tmp-dir\") pod \"55f542a3-ff20-488a-87f3-29219eb39473\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") "
Sep 08 14:50:17.077239 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077216    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-home\") pod \"55f542a3-ff20-488a-87f3-29219eb39473\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") "
Sep 08 14:50:17.077239 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077236    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-dshm\") pod \"55f542a3-ff20-488a-87f3-29219eb39473\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") "
Sep 08 14:50:17.077396 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077286    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grxpc\" (UniqueName: \"kubernetes.io/projected/55f542a3-ff20-488a-87f3-29219eb39473-kube-api-access-grxpc\") pod \"55f542a3-ff20-488a-87f3-29219eb39473\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") "
Sep 08 14:50:17.077396 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077315    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55f542a3-ff20-488a-87f3-29219eb39473-tls-certs\") pod \"55f542a3-ff20-488a-87f3-29219eb39473\" (UID: \"55f542a3-ff20-488a-87f3-29219eb39473\") "
Sep 08 14:50:17.077503 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077406    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "55f542a3-ff20-488a-87f3-29219eb39473" (UID: "55f542a3-ff20-488a-87f3-29219eb39473"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:50:17.077503 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077424    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-model-cache" (OuterVolumeSpecName: "model-cache") pod "55f542a3-ff20-488a-87f3-29219eb39473" (UID: "55f542a3-ff20-488a-87f3-29219eb39473"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:50:17.077503 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077484    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-home" (OuterVolumeSpecName: "home") pod "55f542a3-ff20-488a-87f3-29219eb39473" (UID: "55f542a3-ff20-488a-87f3-29219eb39473"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:50:17.077878 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077854    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:17.078165 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077882    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:17.078165 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.077892    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:17.079686 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.079654    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-dshm" (OuterVolumeSpecName: "dshm") pod "55f542a3-ff20-488a-87f3-29219eb39473" (UID: "55f542a3-ff20-488a-87f3-29219eb39473"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:50:17.079793 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.079718    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f542a3-ff20-488a-87f3-29219eb39473-kube-api-access-grxpc" (OuterVolumeSpecName: "kube-api-access-grxpc") pod "55f542a3-ff20-488a-87f3-29219eb39473" (UID: "55f542a3-ff20-488a-87f3-29219eb39473"). InnerVolumeSpecName "kube-api-access-grxpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:50:17.079793 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.079734    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f542a3-ff20-488a-87f3-29219eb39473-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "55f542a3-ff20-488a-87f3-29219eb39473" (UID: "55f542a3-ff20-488a-87f3-29219eb39473"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:50:17.178643 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.178608    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55f542a3-ff20-488a-87f3-29219eb39473-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:17.178643 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.178635    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grxpc\" (UniqueName: \"kubernetes.io/projected/55f542a3-ff20-488a-87f3-29219eb39473-kube-api-access-grxpc\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:17.178643 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.178649    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55f542a3-ff20-488a-87f3-29219eb39473-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:17.407879 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.407758    2583 generic.go:356] "Generic (PLEG): container finished" podID="55f542a3-ff20-488a-87f3-29219eb39473" containerID="7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff" exitCode=0
Sep 08 14:50:17.407879 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.407813    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"
Sep 08 14:50:17.407879 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.407858    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7" event={"ID":"55f542a3-ff20-488a-87f3-29219eb39473","Type":"ContainerDied","Data":"7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff"}
Sep 08 14:50:17.408171 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.407909    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7" event={"ID":"55f542a3-ff20-488a-87f3-29219eb39473","Type":"ContainerDied","Data":"fd0b4fa63878e8659500b6be13aadcfccebbec56ac03922684335174088d0f50"}
Sep 08 14:50:17.408171 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.407934    2583 scope.go:117] "RemoveContainer" containerID="7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff"
Sep 08 14:50:17.418137 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.418113    2583 scope.go:117] "RemoveContainer" containerID="7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff"
Sep 08 14:50:17.418442 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:50:17.418410    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff\": container with ID starting with 7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff not found: ID does not exist" containerID="7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff"
Sep 08 14:50:17.418538 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.418450    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff"} err="failed to get container status \"7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff\": rpc error: code = NotFound desc = could not find container \"7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff\": container with ID starting with 7c8dd4a49d5b1e3d0a5e5b519f1c799c4b54951bd70dabd69ae0486408ec45ff not found: ID does not exist"
Sep 08 14:50:17.435917 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.435893    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"]
Sep 08 14:50:17.445623 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.445596    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-02a199/test-model-02a199-kserve-f4d8dd885-b88v7"]
Sep 08 14:50:17.795164 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:17.795136    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55f542a3-ff20-488a-87f3-29219eb39473" path="/var/lib/kubelet/pods/55f542a3-ff20-488a-87f3-29219eb39473/volumes"
Sep 08 14:50:18.792148 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:18.792119    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:18.907408 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:18.907376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:19.930816 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:19.930738    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:20.002174 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.002137    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:20.183422 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.183329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:20.495999 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.495966    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"]
Sep 08 14:50:20.496374 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496360    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55f542a3-ff20-488a-87f3-29219eb39473" containerName="main"
Sep 08 14:50:20.496415 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496376    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="55f542a3-ff20-488a-87f3-29219eb39473" containerName="main"
Sep 08 14:50:20.496415 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496389    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ecf62e9-667e-4a12-963e-72f7922639db" containerName="main"
Sep 08 14:50:20.496415 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496395    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ecf62e9-667e-4a12-963e-72f7922639db" containerName="main"
Sep 08 14:50:20.496415 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496404    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d169e6eb-618c-4fb6-b214-164a0a6ffb68" containerName="payload-processing"
Sep 08 14:50:20.496415 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496410    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d169e6eb-618c-4fb6-b214-164a0a6ffb68" containerName="payload-processing"
Sep 08 14:50:20.496573 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496419    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c2b1e9-e3cb-4a65-b016-5e59d1cef488" containerName="payload-pre-processing"
Sep 08 14:50:20.496573 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496424    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c2b1e9-e3cb-4a65-b016-5e59d1cef488" containerName="payload-pre-processing"
Sep 08 14:50:20.496573 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496482    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="64c2b1e9-e3cb-4a65-b016-5e59d1cef488" containerName="payload-pre-processing"
Sep 08 14:50:20.496573 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496491    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ecf62e9-667e-4a12-963e-72f7922639db" containerName="main"
Sep 08 14:50:20.496573 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496498    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d169e6eb-618c-4fb6-b214-164a0a6ffb68" containerName="payload-processing"
Sep 08 14:50:20.496573 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.496503    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="55f542a3-ff20-488a-87f3-29219eb39473" containerName="main"
Sep 08 14:50:20.500800 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.500783    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"
Sep 08 14:50:20.503540 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.503519    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 14:50:20.503540 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.503532    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:50:20.508896 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.508872    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"]
Sep 08 14:50:20.597570 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.597537    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:20.606718 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.606696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm\" (UID: \"a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"
Sep 08 14:50:20.707874 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.707814    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm\" (UID: \"a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"
Sep 08 14:50:20.708414 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.708393    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm\" (UID: \"a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"
Sep 08 14:50:20.813304 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:20.813231    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"
Sep 08 14:50:21.090501 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:21.090479    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"]
Sep 08 14:50:21.092705 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:50:21.092679    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda011ee4d_e78d_4b2f_ab2b_eeabf83b2a3d.slice/crio-bc6937957f68740b6d8a9c2951273ea175ab068678ee2a59315872c3a7c75e4d WatchSource:0}: Error finding container bc6937957f68740b6d8a9c2951273ea175ab068678ee2a59315872c3a7c75e4d: Status 404 returned error can't find the container with id bc6937957f68740b6d8a9c2951273ea175ab068678ee2a59315872c3a7c75e4d
Sep 08 14:50:21.428335 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:21.428301    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm" event={"ID":"a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d","Type":"ContainerStarted","Data":"1d22cc038f824a94f324b91d1c01a1bd314aaeb9d56af532e09dfe05c49d6e2f"}
Sep 08 14:50:21.428468 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:21.428352    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm" event={"ID":"a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d","Type":"ContainerStarted","Data":"bc6937957f68740b6d8a9c2951273ea175ab068678ee2a59315872c3a7c75e4d"}
Sep 08 14:50:21.450114 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:21.450070    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm" podStartSLOduration=1.450054857 podStartE2EDuration="1.450054857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:50:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:50:21.448030917 +0000 UTC m=+972.175847932" watchObservedRunningTime="2026-09-08 14:50:21.450054857 +0000 UTC m=+972.177871870"
Sep 08 14:50:22.435801 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:22.435756    2583 generic.go:356] "Generic (PLEG): container finished" podID="a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d" containerID="1d22cc038f824a94f324b91d1c01a1bd314aaeb9d56af532e09dfe05c49d6e2f" exitCode=0
Sep 08 14:50:22.436255 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:22.435860    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm" event={"ID":"a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d","Type":"ContainerDied","Data":"1d22cc038f824a94f324b91d1c01a1bd314aaeb9d56af532e09dfe05c49d6e2f"}
Sep 08 14:50:23.584475 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:23.584451    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"
Sep 08 14:50:23.635566 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:23.635535    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d-maas-api-service-ca\") pod \"a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d\" (UID: \"a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d\") "
Sep 08 14:50:23.635884 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:23.635863    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d" (UID: "a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:50:23.737106 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:23.737079    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:24.449107 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:24.449074    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"
Sep 08 14:50:24.449281 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:24.449114    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm" event={"ID":"a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d","Type":"ContainerDied","Data":"bc6937957f68740b6d8a9c2951273ea175ab068678ee2a59315872c3a7c75e4d"}
Sep 08 14:50:24.449281 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:24.449158    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc6937957f68740b6d8a9c2951273ea175ab068678ee2a59315872c3a7c75e4d"
Sep 08 14:50:31.351274 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.351190    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:31.390795 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.390759    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"]
Sep 08 14:50:31.391036 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.391005    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5" podUID="f3b9ccdd-97a5-4095-a0d9-c299350db2be" containerName="payload-pre-processing" containerID="cri-o://41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef" gracePeriod=30
Sep 08 14:50:31.432439 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.432392    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:31.542350 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.542315    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:31.582610 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.582555    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"]
Sep 08 14:50:31.585409 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.585378    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-02a199-bq2zm"]
Sep 08 14:50:31.642019 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.641992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:31.647809 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.647790    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:50:31.706640 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.706613    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3b9ccdd-97a5-4095-a0d9-c299350db2be-plugins-config-volume\") pod \"f3b9ccdd-97a5-4095-a0d9-c299350db2be\" (UID: \"f3b9ccdd-97a5-4095-a0d9-c299350db2be\") "
Sep 08 14:50:31.706779 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.706734    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgfv5\" (UniqueName: \"kubernetes.io/projected/f3b9ccdd-97a5-4095-a0d9-c299350db2be-kube-api-access-rgfv5\") pod \"f3b9ccdd-97a5-4095-a0d9-c299350db2be\" (UID: \"f3b9ccdd-97a5-4095-a0d9-c299350db2be\") "
Sep 08 14:50:31.707066 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.707043    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3b9ccdd-97a5-4095-a0d9-c299350db2be-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f3b9ccdd-97a5-4095-a0d9-c299350db2be" (UID: "f3b9ccdd-97a5-4095-a0d9-c299350db2be"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:50:31.708941 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.708918    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3b9ccdd-97a5-4095-a0d9-c299350db2be-kube-api-access-rgfv5" (OuterVolumeSpecName: "kube-api-access-rgfv5") pod "f3b9ccdd-97a5-4095-a0d9-c299350db2be" (UID: "f3b9ccdd-97a5-4095-a0d9-c299350db2be"). InnerVolumeSpecName "kube-api-access-rgfv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:50:31.793733 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.793705    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d" path="/var/lib/kubelet/pods/a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d/volumes"
Sep 08 14:50:31.808070 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.808048    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3b9ccdd-97a5-4095-a0d9-c299350db2be-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:31.808070 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.808070    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgfv5\" (UniqueName: \"kubernetes.io/projected/f3b9ccdd-97a5-4095-a0d9-c299350db2be-kube-api-access-rgfv5\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:50:31.956892 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:31.956807    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:32.271139 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:32.271092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:32.498055 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:32.498027    2583 generic.go:356] "Generic (PLEG): container finished" podID="f3b9ccdd-97a5-4095-a0d9-c299350db2be" containerID="41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef" exitCode=0
Sep 08 14:50:32.498459 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:32.498070    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"
Sep 08 14:50:32.498459 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:32.498121    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5" event={"ID":"f3b9ccdd-97a5-4095-a0d9-c299350db2be","Type":"ContainerDied","Data":"41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef"}
Sep 08 14:50:32.498459 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:32.498168    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5" event={"ID":"f3b9ccdd-97a5-4095-a0d9-c299350db2be","Type":"ContainerDied","Data":"fbe316f52dacaf5f08d6d4b6397c1c91709e5699e2579b82763c8c455d7a1759"}
Sep 08 14:50:32.498459 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:32.498194    2583 scope.go:117] "RemoveContainer" containerID="41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef"
Sep 08 14:50:32.506889 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:32.506873    2583 scope.go:117] "RemoveContainer" containerID="41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef"
Sep 08 14:50:32.507157 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:50:32.507139    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef\": container with ID starting with 41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef not found: ID does not exist" containerID="41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef"
Sep 08 14:50:32.507207 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:32.507166    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef"} err="failed to get container status \"41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef\": rpc error: code = NotFound desc = could not find container \"41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef\": container with ID starting with 41efc1975dfbea2da46516591cba0df5b550adaefdf85dcde4bba9428ee1a9ef not found: ID does not exist"
Sep 08 14:50:32.517202 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:32.517179    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"]
Sep 08 14:50:32.523679 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:32.523628    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-02a199-574d6668d6-j8rm5"]
Sep 08 14:50:33.792957 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:33.792926    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3b9ccdd-97a5-4095-a0d9-c299350db2be" path="/var/lib/kubelet/pods/f3b9ccdd-97a5-4095-a0d9-c299350db2be/volumes"
Sep 08 14:50:37.889776 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:37.889743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:38.072088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:38.072015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:38.173691 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:38.173617    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:38.348103 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:38.348069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:38.906151 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:38.906115    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:39.269070 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:39.269037    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:39.470750 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:39.470711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:40.525443 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:40.525413    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:41.209891 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:41.209857    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:41.302056 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:41.302015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:41.608679 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:41.608646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:42.306608 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:42.306577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:42.397422 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:42.397393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:43.694681 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:43.694640    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:43.791463 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:43.791424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:43.890898 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:43.890862    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:44.291710 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.291679    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:44.392280 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.392250    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:44.474743 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.474700    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"]
Sep 08 14:50:44.475130 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.475115    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3b9ccdd-97a5-4095-a0d9-c299350db2be" containerName="payload-pre-processing"
Sep 08 14:50:44.475176 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.475132    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3b9ccdd-97a5-4095-a0d9-c299350db2be" containerName="payload-pre-processing"
Sep 08 14:50:44.475176 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.475154    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d" containerName="revoke-keys"
Sep 08 14:50:44.475176 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.475162    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d" containerName="revoke-keys"
Sep 08 14:50:44.475272 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.475222    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3b9ccdd-97a5-4095-a0d9-c299350db2be" containerName="payload-pre-processing"
Sep 08 14:50:44.475272 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.475231    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a011ee4d-e78d-4b2f-ab2b-eeabf83b2a3d" containerName="revoke-keys"
Sep 08 14:50:44.478373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.478358    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 14:50:44.481121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.481099    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-ef8f61\""
Sep 08 14:50:44.481650 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.481636    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-ef8f61-dockercfg-fjhjj\""
Sep 08 14:50:44.490435 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.490412    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"]
Sep 08 14:50:44.627700 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.627623    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s78sc\" (UniqueName: \"kubernetes.io/projected/638d0163-dc0b-45ea-8566-7b5ced202652-kube-api-access-s78sc\") pod \"payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl\" (UID: \"638d0163-dc0b-45ea-8566-7b5ced202652\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 14:50:44.627700 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.627662    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/638d0163-dc0b-45ea-8566-7b5ced202652-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl\" (UID: \"638d0163-dc0b-45ea-8566-7b5ced202652\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 14:50:44.728917 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.728877    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s78sc\" (UniqueName: \"kubernetes.io/projected/638d0163-dc0b-45ea-8566-7b5ced202652-kube-api-access-s78sc\") pod \"payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl\" (UID: \"638d0163-dc0b-45ea-8566-7b5ced202652\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 14:50:44.729319 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.728925    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/638d0163-dc0b-45ea-8566-7b5ced202652-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl\" (UID: \"638d0163-dc0b-45ea-8566-7b5ced202652\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 14:50:44.729577 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.729558    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/638d0163-dc0b-45ea-8566-7b5ced202652-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl\" (UID: \"638d0163-dc0b-45ea-8566-7b5ced202652\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 14:50:44.738594 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.738565    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s78sc\" (UniqueName: \"kubernetes.io/projected/638d0163-dc0b-45ea-8566-7b5ced202652-kube-api-access-s78sc\") pod \"payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl\" (UID: \"638d0163-dc0b-45ea-8566-7b5ced202652\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 14:50:44.789142 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:44.789116    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 14:50:45.064232 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:45.064208    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"]
Sep 08 14:50:45.066093 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:50:45.066068    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod638d0163_dc0b_45ea_8566_7b5ced202652.slice/crio-1a0528001603e208c378a79f88b59c4e383e9c34a3bd4741e19e883c83f61662 WatchSource:0}: Error finding container 1a0528001603e208c378a79f88b59c4e383e9c34a3bd4741e19e883c83f61662: Status 404 returned error can't find the container with id 1a0528001603e208c378a79f88b59c4e383e9c34a3bd4741e19e883c83f61662
Sep 08 14:50:45.566021 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:45.565983    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl" event={"ID":"638d0163-dc0b-45ea-8566-7b5ced202652","Type":"ContainerStarted","Data":"88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8"}
Sep 08 14:50:45.566202 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:45.566041    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl" event={"ID":"638d0163-dc0b-45ea-8566-7b5ced202652","Type":"ContainerStarted","Data":"1a0528001603e208c378a79f88b59c4e383e9c34a3bd4741e19e883c83f61662"}
Sep 08 14:50:45.566202 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:45.566069    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 14:50:48.058224 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:48.058189    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:48.159938 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:48.159853    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:48.594109 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:48.594069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:56.573606 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:56.573574    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 14:50:56.592775 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:56.592722    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl" podStartSLOduration=12.59270723 podStartE2EDuration="12.59270723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:50:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:50:45.583182823 +0000 UTC m=+996.310999838" watchObservedRunningTime="2026-09-08 14:50:56.59270723 +0000 UTC m=+1007.320524244"
Sep 08 14:50:58.930550 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:58.930520    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:59.028186 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:59.028150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:59.112720 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:59.112687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:59.184092 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:59.184020    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:50:59.486400 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:50:59.486367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:04.704140 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:04.704099    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:05.677338 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:05.677306    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:05.776233 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:05.776201    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:05.958420 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:05.958338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:06.367770 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:06.367735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:06.543198 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:06.543164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:06.644391 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:06.644287    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:07.167779 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:07.167744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:08.373293 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:08.373256    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:08.474904 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:08.474872    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:10.918948 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:10.918911    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:11.018955 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:11.018921    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:12.204067 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:12.204028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:12.613123 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:12.613090    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:12.718196 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:12.718162    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:12.822560 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:12.822526    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:12.915678 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:12.915598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:13.560924 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:13.560882    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:13.673109 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:13.673075    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:13.775361 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:13.775335    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:19.768632 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:19.768598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:19.855794 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:19.855761    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:20.679277 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:20.679232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:21.010067 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:21.010029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:21.220940 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:21.220893    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:24.490437 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:24.490404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:24.580220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:24.580176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:25.631542 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:25.631513    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:25.701776 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:25.701739    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:26.122077 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.122040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:26.222934 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.222890    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:26.238771 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.238739    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"]
Sep 08 14:51:26.243723 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.243702    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:51:26.246587 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.246563    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-11dfb4\""
Sep 08 14:51:26.256754 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.256732    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"]
Sep 08 14:51:26.262870 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.262845    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"]
Sep 08 14:51:26.266144 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.266128    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:51:26.269115 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.269097    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-11dfb4-dockercfg-ptdp2\""
Sep 08 14:51:26.283809 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.283782    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"]
Sep 08 14:51:26.293649 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.293625    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fe1e20-f93f-4b50-9699-a61661df4b52-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p\" (UID: \"f5fe1e20-f93f-4b50-9699-a61661df4b52\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:51:26.293784 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.293693    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltlx2\" (UniqueName: \"kubernetes.io/projected/f5fe1e20-f93f-4b50-9699-a61661df4b52-kube-api-access-ltlx2\") pod \"payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p\" (UID: \"f5fe1e20-f93f-4b50-9699-a61661df4b52\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:51:26.394231 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.394155    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fe1e20-f93f-4b50-9699-a61661df4b52-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p\" (UID: \"f5fe1e20-f93f-4b50-9699-a61661df4b52\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:51:26.394231 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.394192    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlb8z\" (UniqueName: \"kubernetes.io/projected/a0277548-02bb-4266-96cb-9fd882ffe3dc-kube-api-access-rlb8z\") pod \"payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l\" (UID: \"a0277548-02bb-4266-96cb-9fd882ffe3dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:51:26.394435 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.394240    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0277548-02bb-4266-96cb-9fd882ffe3dc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l\" (UID: \"a0277548-02bb-4266-96cb-9fd882ffe3dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:51:26.394435 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.394288    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltlx2\" (UniqueName: \"kubernetes.io/projected/f5fe1e20-f93f-4b50-9699-a61661df4b52-kube-api-access-ltlx2\") pod \"payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p\" (UID: \"f5fe1e20-f93f-4b50-9699-a61661df4b52\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:51:26.394824 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.394804    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fe1e20-f93f-4b50-9699-a61661df4b52-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p\" (UID: \"f5fe1e20-f93f-4b50-9699-a61661df4b52\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:51:26.408190 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.408170    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltlx2\" (UniqueName: \"kubernetes.io/projected/f5fe1e20-f93f-4b50-9699-a61661df4b52-kube-api-access-ltlx2\") pod \"payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p\" (UID: \"f5fe1e20-f93f-4b50-9699-a61661df4b52\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:51:26.496001 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.495971    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlb8z\" (UniqueName: \"kubernetes.io/projected/a0277548-02bb-4266-96cb-9fd882ffe3dc-kube-api-access-rlb8z\") pod \"payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l\" (UID: \"a0277548-02bb-4266-96cb-9fd882ffe3dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:51:26.496135 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.496018    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0277548-02bb-4266-96cb-9fd882ffe3dc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l\" (UID: \"a0277548-02bb-4266-96cb-9fd882ffe3dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:51:26.496599 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.496582    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0277548-02bb-4266-96cb-9fd882ffe3dc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l\" (UID: \"a0277548-02bb-4266-96cb-9fd882ffe3dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:51:26.509486 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.509465    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlb8z\" (UniqueName: \"kubernetes.io/projected/a0277548-02bb-4266-96cb-9fd882ffe3dc-kube-api-access-rlb8z\") pod \"payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l\" (UID: \"a0277548-02bb-4266-96cb-9fd882ffe3dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:51:26.554129 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.554098    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:51:26.580655 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.580621    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:51:26.829266 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.829241    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"]
Sep 08 14:51:26.831101 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:51:26.831067    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5fe1e20_f93f_4b50_9699_a61661df4b52.slice/crio-c08dccb57c93535a52c2f0a65904d6153447758ffe50e5296ff83473dd4012c7 WatchSource:0}: Error finding container c08dccb57c93535a52c2f0a65904d6153447758ffe50e5296ff83473dd4012c7: Status 404 returned error can't find the container with id c08dccb57c93535a52c2f0a65904d6153447758ffe50e5296ff83473dd4012c7
Sep 08 14:51:26.867465 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:26.867440    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"]
Sep 08 14:51:26.868871 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:51:26.868844    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0277548_02bb_4266_96cb_9fd882ffe3dc.slice/crio-8654266df3d15074539f347cbc2ff82db508bd8d539cd12f478c0893e694ca9b WatchSource:0}: Error finding container 8654266df3d15074539f347cbc2ff82db508bd8d539cd12f478c0893e694ca9b: Status 404 returned error can't find the container with id 8654266df3d15074539f347cbc2ff82db508bd8d539cd12f478c0893e694ca9b
Sep 08 14:51:27.772173 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:27.772136    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p" event={"ID":"f5fe1e20-f93f-4b50-9699-a61661df4b52","Type":"ContainerStarted","Data":"f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef"}
Sep 08 14:51:27.772376 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:27.772201    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p" event={"ID":"f5fe1e20-f93f-4b50-9699-a61661df4b52","Type":"ContainerStarted","Data":"c08dccb57c93535a52c2f0a65904d6153447758ffe50e5296ff83473dd4012c7"}
Sep 08 14:51:27.772376 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:27.772231    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:51:27.773731 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:27.773709    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l" event={"ID":"a0277548-02bb-4266-96cb-9fd882ffe3dc","Type":"ContainerStarted","Data":"e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5"}
Sep 08 14:51:27.773825 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:27.773753    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l" event={"ID":"a0277548-02bb-4266-96cb-9fd882ffe3dc","Type":"ContainerStarted","Data":"8654266df3d15074539f347cbc2ff82db508bd8d539cd12f478c0893e694ca9b"}
Sep 08 14:51:27.773909 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:27.773856    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:51:27.790884 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:27.790822    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p" podStartSLOduration=1.790802898 podStartE2EDuration="1.790802898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:51:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:51:27.789201346 +0000 UTC m=+1038.517018376" watchObservedRunningTime="2026-09-08 14:51:27.790802898 +0000 UTC m=+1038.518619917"
Sep 08 14:51:27.810784 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:27.810729    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l" podStartSLOduration=1.810710561 podStartE2EDuration="1.810710561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:51:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:51:27.808014757 +0000 UTC m=+1038.535831773" watchObservedRunningTime="2026-09-08 14:51:27.810710561 +0000 UTC m=+1038.538527577"
Sep 08 14:51:33.214987 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:33.214955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:35.860594 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:35.860553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:38.781865 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:38.781819    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:51:38.782285 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:38.782017    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:51:44.668977 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.668928    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"]
Sep 08 14:51:44.674864 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.674810    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.677912 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.677884    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ef8f61\"/\"kube-root-ca.crt\""
Sep 08 14:51:44.679189 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.679164    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ef8f61\"/\"default-dockercfg-ffnc6\""
Sep 08 14:51:44.679298 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.679170    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ef8f61\"/\"auth-test-model-ef8f61-kserve-self-signed-certs\""
Sep 08 14:51:44.679298 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.679171    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ef8f61\"/\"openshift-service-ca.crt\""
Sep 08 14:51:44.683601 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.683528    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"]
Sep 08 14:51:44.761629 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.761594    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-dshm\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.761790 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.761636    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-home\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.761790 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.761706    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-tmp-dir\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.761790 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.761777    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-model-cache\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.761959 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.761805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxmpf\" (UniqueName: \"kubernetes.io/projected/f15ecc19-37d0-4591-8f68-5252cf61ab45-kube-api-access-hxmpf\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.761959 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.761880    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15ecc19-37d0-4591-8f68-5252cf61ab45-tls-certs\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.862623 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.862591    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-tmp-dir\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.862806 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.862648    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-model-cache\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.862806 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.862671    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxmpf\" (UniqueName: \"kubernetes.io/projected/f15ecc19-37d0-4591-8f68-5252cf61ab45-kube-api-access-hxmpf\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.862806 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.862724    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15ecc19-37d0-4591-8f68-5252cf61ab45-tls-certs\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.863072 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.863046    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-dshm\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.863259 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.863236    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-home\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.863484 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.863109    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-model-cache\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.863569 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.863085    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-tmp-dir\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.863569 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.863495    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-home\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.863861 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.863822    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:44.865333 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.865310    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-dshm\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.865583 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.865563    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15ecc19-37d0-4591-8f68-5252cf61ab45-tls-certs\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.871702 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.871682    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxmpf\" (UniqueName: \"kubernetes.io/projected/f15ecc19-37d0-4591-8f68-5252cf61ab45-kube-api-access-hxmpf\") pod \"auth-test-model-ef8f61-kserve-5b468d654c-v9dtm\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:44.988330 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:44.988310    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:45.264200 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:45.264172    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"]
Sep 08 14:51:45.266137 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:51:45.266109    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf15ecc19_37d0_4591_8f68_5252cf61ab45.slice/crio-1fb607640d50d1eb98525dd59412d1d2cfa7c836cb7c9356efaa211cfc1ac25a WatchSource:0}: Error finding container 1fb607640d50d1eb98525dd59412d1d2cfa7c836cb7c9356efaa211cfc1ac25a: Status 404 returned error can't find the container with id 1fb607640d50d1eb98525dd59412d1d2cfa7c836cb7c9356efaa211cfc1ac25a
Sep 08 14:51:45.871460 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:45.871426    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm" event={"ID":"f15ecc19-37d0-4591-8f68-5252cf61ab45","Type":"ContainerStarted","Data":"057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150"}
Sep 08 14:51:45.871847 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:45.871470    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm" event={"ID":"f15ecc19-37d0-4591-8f68-5252cf61ab45","Type":"ContainerStarted","Data":"1fb607640d50d1eb98525dd59412d1d2cfa7c836cb7c9356efaa211cfc1ac25a"}
Sep 08 14:51:45.891907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:45.891862    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm" podStartSLOduration=1.891846095 podStartE2EDuration="1.891846095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:51:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:51:45.889608366 +0000 UTC m=+1056.617425380" watchObservedRunningTime="2026-09-08 14:51:45.891846095 +0000 UTC m=+1056.619663099"
Sep 08 14:51:46.554037 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:46.554000    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:49.447397 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:49.447351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:49.781369 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:49.781336    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:50.098930 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:50.098820    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:50.364070 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:50.363983    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:53.201004 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:53.200965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:53.288007 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:53.287978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:54.335816 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:54.335779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:54.438291 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:54.438258    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:54.862592 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:54.862563    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:54.962016 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:54.961988    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:54.989236 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:54.989202    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:54.989382 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:54.989245    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:55.002026 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:55.002005    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:51:55.339315 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:55.339273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:51:55.932649 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:51:55.932619    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:52:14.638334 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:14.638295    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:14.759228 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:14.759198    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:18.197785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:18.197751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:21.827695 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:21.827659    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7868-139679228187968"]
Sep 08 14:52:21.831460 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:21.831443    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7868-139679228187968"
Sep 08 14:52:21.834719 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:21.834696    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2sfkk\""
Sep 08 14:52:21.845245 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:21.845226    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7868-139679228187968"]
Sep 08 14:52:21.885992 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:21.885967    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpqwl\" (UniqueName: \"kubernetes.io/projected/26410996-5c4a-4b0d-b029-f93548d711e7-kube-api-access-wpqwl\") pod \"test-curl-7868-139679228187968\" (UID: \"26410996-5c4a-4b0d-b029-f93548d711e7\") " pod="opendatahub/test-curl-7868-139679228187968"
Sep 08 14:52:21.987379 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:21.987341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpqwl\" (UniqueName: \"kubernetes.io/projected/26410996-5c4a-4b0d-b029-f93548d711e7-kube-api-access-wpqwl\") pod \"test-curl-7868-139679228187968\" (UID: \"26410996-5c4a-4b0d-b029-f93548d711e7\") " pod="opendatahub/test-curl-7868-139679228187968"
Sep 08 14:52:21.997680 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:21.997646    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpqwl\" (UniqueName: \"kubernetes.io/projected/26410996-5c4a-4b0d-b029-f93548d711e7-kube-api-access-wpqwl\") pod \"test-curl-7868-139679228187968\" (UID: \"26410996-5c4a-4b0d-b029-f93548d711e7\") " pod="opendatahub/test-curl-7868-139679228187968"
Sep 08 14:52:22.140892 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:22.140771    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7868-139679228187968"
Sep 08 14:52:22.420122 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:22.420038    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7868-139679228187968"]
Sep 08 14:52:22.423786 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:52:22.423754    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26410996_5c4a_4b0d_b029_f93548d711e7.slice/crio-3641b66aced18918bacb2931c7465c70a7cb03d4c65858aaaadf0ac5bcdc224a WatchSource:0}: Error finding container 3641b66aced18918bacb2931c7465c70a7cb03d4c65858aaaadf0ac5bcdc224a: Status 404 returned error can't find the container with id 3641b66aced18918bacb2931c7465c70a7cb03d4c65858aaaadf0ac5bcdc224a
Sep 08 14:52:22.425184 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:22.425164    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:52:23.056788 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:23.056748    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7868-139679228187968" event={"ID":"26410996-5c4a-4b0d-b029-f93548d711e7","Type":"ContainerStarted","Data":"3641b66aced18918bacb2931c7465c70a7cb03d4c65858aaaadf0ac5bcdc224a"}
Sep 08 14:52:24.965783 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:24.965747    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"]
Sep 08 14:52:24.969801 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:24.969781    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:24.973961 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:24.973940    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"openshift-service-ca.crt\""
Sep 08 14:52:24.975016 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:24.974999    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"kube-root-ca.crt\""
Sep 08 14:52:24.975122 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:24.975040    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"auth-test-model-f83533-kserve-self-signed-certs\""
Sep 08 14:52:24.975208 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:24.975190    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"default-dockercfg-mxzdm\""
Sep 08 14:52:24.979099 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:24.979074    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"]
Sep 08 14:52:25.002768 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.002740    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:25.075420 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.075384    2583 generic.go:356] "Generic (PLEG): container finished" podID="26410996-5c4a-4b0d-b029-f93548d711e7" containerID="820c1b28fac357e0122374ddf9edad493e262b259342df817ab15230651fe19c" exitCode=0
Sep 08 14:52:25.075581 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.075442    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7868-139679228187968" event={"ID":"26410996-5c4a-4b0d-b029-f93548d711e7","Type":"ContainerDied","Data":"820c1b28fac357e0122374ddf9edad493e262b259342df817ab15230651fe19c"}
Sep 08 14:52:25.119581 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.119555    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-home\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.119744 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.119598    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a687bf9-2a16-4bb7-9f22-a254d183561d-tls-certs\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.119744 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.119715    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-model-cache\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.119907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.119796    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9khgp\" (UniqueName: \"kubernetes.io/projected/9a687bf9-2a16-4bb7-9f22-a254d183561d-kube-api-access-9khgp\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.119907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.119854    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-dshm\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.119907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.119887    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-tmp-dir\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.220518 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.220489    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a687bf9-2a16-4bb7-9f22-a254d183561d-tls-certs\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.220669 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.220543    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-model-cache\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.220669 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.220585    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9khgp\" (UniqueName: \"kubernetes.io/projected/9a687bf9-2a16-4bb7-9f22-a254d183561d-kube-api-access-9khgp\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.220669 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.220604    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-dshm\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.220669 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.220619    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-tmp-dir\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.220669 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.220655    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-home\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.221120 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.221093    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-tmp-dir\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.221120 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.221106    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-home\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.221263 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.221215    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-model-cache\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.223057 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.223027    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-dshm\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.223273 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.223252    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a687bf9-2a16-4bb7-9f22-a254d183561d-tls-certs\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.231785 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.231762    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9khgp\" (UniqueName: \"kubernetes.io/projected/9a687bf9-2a16-4bb7-9f22-a254d183561d-kube-api-access-9khgp\") pod \"auth-test-model-f83533-kserve-77789dbc55-whp8z\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") " pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.280394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.280369    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:25.554675 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:25.554652    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"]
Sep 08 14:52:25.556317 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:52:25.556295    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a687bf9_2a16_4bb7_9f22_a254d183561d.slice/crio-7db001eef8430d0e090812bce11abef10effd9a60c94ab478692e78e1a135860 WatchSource:0}: Error finding container 7db001eef8430d0e090812bce11abef10effd9a60c94ab478692e78e1a135860: Status 404 returned error can't find the container with id 7db001eef8430d0e090812bce11abef10effd9a60c94ab478692e78e1a135860
Sep 08 14:52:26.085300 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:26.085265    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z" event={"ID":"9a687bf9-2a16-4bb7-9f22-a254d183561d","Type":"ContainerStarted","Data":"55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7"}
Sep 08 14:52:26.085702 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:26.085313    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z" event={"ID":"9a687bf9-2a16-4bb7-9f22-a254d183561d","Type":"ContainerStarted","Data":"7db001eef8430d0e090812bce11abef10effd9a60c94ab478692e78e1a135860"}
Sep 08 14:52:26.105724 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:26.105651    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z" podStartSLOduration=2.105627283 podStartE2EDuration="2.105627283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:52:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:52:26.10389583 +0000 UTC m=+1096.831712845" watchObservedRunningTime="2026-09-08 14:52:26.105627283 +0000 UTC m=+1096.833444361"
Sep 08 14:52:26.211821 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:26.211798    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7868-139679228187968"
Sep 08 14:52:26.232424 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:26.232400    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7868-139679228187968_26410996-5c4a-4b0d-b029-f93548d711e7/test-curl-7868-139679228187968/0.log"
Sep 08 14:52:26.257121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:26.257090    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7868-139679228187968"]
Sep 08 14:52:26.260677 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:26.260653    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7868-139679228187968"]
Sep 08 14:52:26.330309 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:26.330284    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpqwl\" (UniqueName: \"kubernetes.io/projected/26410996-5c4a-4b0d-b029-f93548d711e7-kube-api-access-wpqwl\") pod \"26410996-5c4a-4b0d-b029-f93548d711e7\" (UID: \"26410996-5c4a-4b0d-b029-f93548d711e7\") "
Sep 08 14:52:26.332543 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:26.332514    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26410996-5c4a-4b0d-b029-f93548d711e7-kube-api-access-wpqwl" (OuterVolumeSpecName: "kube-api-access-wpqwl") pod "26410996-5c4a-4b0d-b029-f93548d711e7" (UID: "26410996-5c4a-4b0d-b029-f93548d711e7"). InnerVolumeSpecName "kube-api-access-wpqwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:52:26.431400 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:26.431372    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpqwl\" (UniqueName: \"kubernetes.io/projected/26410996-5c4a-4b0d-b029-f93548d711e7-kube-api-access-wpqwl\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:52:27.091783 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:27.091753    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7868-139679228187968"
Sep 08 14:52:27.092186 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:27.091801    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3641b66aced18918bacb2931c7465c70a7cb03d4c65858aaaadf0ac5bcdc224a"
Sep 08 14:52:27.102099 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:52:27.102074    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7868-139679228187968\" is forbidden: User \"system:node:ip-10-0-137-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-6.ec2.internal' and this object" podUID="26410996-5c4a-4b0d-b029-f93548d711e7" pod="opendatahub/test-curl-7868-139679228187968"
Sep 08 14:52:27.793401 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:27.793366    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26410996-5c4a-4b0d-b029-f93548d711e7" path="/var/lib/kubelet/pods/26410996-5c4a-4b0d-b029-f93548d711e7/volumes"
Sep 08 14:52:33.399266 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:33.399234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:33.723160 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:33.723126    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:33.943583 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:33.943549    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:35.281485 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:35.281449    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:35.281899 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:35.281503    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:35.293919 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:35.293897    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:36.150029 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:36.150001    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:52:36.923737 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:36.923702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:37.083579 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:37.083542    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:38.028762 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.028729    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:38.107241 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.107210    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:38.207038 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.206998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:38.550450 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.550423    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:38.650480 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.650447    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:52:38.671289 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.671260    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"]
Sep 08 14:52:38.671645 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.671633    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26410996-5c4a-4b0d-b029-f93548d711e7" containerName="test-curl-7868-139679228187968"
Sep 08 14:52:38.671700 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.671647    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="26410996-5c4a-4b0d-b029-f93548d711e7" containerName="test-curl-7868-139679228187968"
Sep 08 14:52:38.671741 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.671716    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="26410996-5c4a-4b0d-b029-f93548d711e7" containerName="test-curl-7868-139679228187968"
Sep 08 14:52:38.674647 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.674631    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:52:38.677866 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.677850    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-3bb80f8b-dockercfg-6vdnr\""
Sep 08 14:52:38.678141 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.678127    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-3bb80f8b\""
Sep 08 14:52:38.692170 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.692148    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"]
Sep 08 14:52:38.850905 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.850797    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/155572f0-9d1d-4724-a282-e63fb60c9472-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss\" (UID: \"155572f0-9d1d-4724-a282-e63fb60c9472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:52:38.851055 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.850967    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgd6z\" (UniqueName: \"kubernetes.io/projected/155572f0-9d1d-4724-a282-e63fb60c9472-kube-api-access-dgd6z\") pod \"payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss\" (UID: \"155572f0-9d1d-4724-a282-e63fb60c9472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:52:38.952091 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.952057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgd6z\" (UniqueName: \"kubernetes.io/projected/155572f0-9d1d-4724-a282-e63fb60c9472-kube-api-access-dgd6z\") pod \"payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss\" (UID: \"155572f0-9d1d-4724-a282-e63fb60c9472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:52:38.952252 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.952115    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/155572f0-9d1d-4724-a282-e63fb60c9472-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss\" (UID: \"155572f0-9d1d-4724-a282-e63fb60c9472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:52:38.952793 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.952773    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/155572f0-9d1d-4724-a282-e63fb60c9472-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss\" (UID: \"155572f0-9d1d-4724-a282-e63fb60c9472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:52:38.960655 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.960626    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgd6z\" (UniqueName: \"kubernetes.io/projected/155572f0-9d1d-4724-a282-e63fb60c9472-kube-api-access-dgd6z\") pod \"payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss\" (UID: \"155572f0-9d1d-4724-a282-e63fb60c9472\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:52:38.985195 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:38.985171    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:52:39.262374 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:39.262346    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"]
Sep 08 14:52:39.264134 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:52:39.264107    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod155572f0_9d1d_4724_a282_e63fb60c9472.slice/crio-f97b9c67c9467c113eb80a3c352b0d9c3b29829218fd1d66187e08af17235d3c WatchSource:0}: Error finding container f97b9c67c9467c113eb80a3c352b0d9c3b29829218fd1d66187e08af17235d3c: Status 404 returned error can't find the container with id f97b9c67c9467c113eb80a3c352b0d9c3b29829218fd1d66187e08af17235d3c
Sep 08 14:52:40.160333 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:40.160295    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss" event={"ID":"155572f0-9d1d-4724-a282-e63fb60c9472","Type":"ContainerStarted","Data":"cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3"}
Sep 08 14:52:40.160533 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:40.160349    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss" event={"ID":"155572f0-9d1d-4724-a282-e63fb60c9472","Type":"ContainerStarted","Data":"f97b9c67c9467c113eb80a3c352b0d9c3b29829218fd1d66187e08af17235d3c"}
Sep 08 14:52:40.160533 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:40.160409    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:52:40.180538 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:40.180486    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss" podStartSLOduration=2.180469748 podStartE2EDuration="2.180469748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:52:40.179131184 +0000 UTC m=+1110.906948199" watchObservedRunningTime="2026-09-08 14:52:40.180469748 +0000 UTC m=+1110.908286816"
Sep 08 14:52:51.167399 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:52:51.167364    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:53:01.827094 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:01.827055    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:03.203718 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:03.203676    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:04.365322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.365289    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"]
Sep 08 14:53:04.373801 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.373780    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.377954 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.377923    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3bb80f8b\"/\"default-dockercfg-kgdth\""
Sep 08 14:53:04.378078 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.377952    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3bb80f8b\"/\"openshift-service-ca.crt\""
Sep 08 14:53:04.378078 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.378007    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3bb80f8b\"/\"e2e-lifecycle-model-3bb80f8b-kserve-self-signed-certs\""
Sep 08 14:53:04.378247 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.378110    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3bb80f8b\"/\"kube-root-ca.crt\""
Sep 08 14:53:04.381713 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.381692    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"]
Sep 08 14:53:04.397771 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.397747    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:04.479885 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.479820    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-tmp-dir\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.480027 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.479903    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5039c9e0-3587-4249-bcaf-26fec07c8527-tls-certs\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.480027 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.479950    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-dshm\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.480027 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.480004    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-model-cache\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.480149 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.480027    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28767\" (UniqueName: \"kubernetes.io/projected/5039c9e0-3587-4249-bcaf-26fec07c8527-kube-api-access-28767\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.480149 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.480076    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-home\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.580723 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.580691    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-model-cache\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.580723 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.580724    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28767\" (UniqueName: \"kubernetes.io/projected/5039c9e0-3587-4249-bcaf-26fec07c8527-kube-api-access-28767\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.580945 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.580760    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-home\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.580945 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.580822    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-tmp-dir\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.581042 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.580963    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5039c9e0-3587-4249-bcaf-26fec07c8527-tls-certs\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.581042 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.581008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-dshm\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.581137 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.581122    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-model-cache\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.581197 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.581181    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-tmp-dir\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.581256 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.581237    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-home\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.583343 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.583318    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-dshm\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.583626 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.583611    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5039c9e0-3587-4249-bcaf-26fec07c8527-tls-certs\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.592676 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.592656    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28767\" (UniqueName: \"kubernetes.io/projected/5039c9e0-3587-4249-bcaf-26fec07c8527-kube-api-access-28767\") pod \"e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") " pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.686442 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.686368    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:04.694158 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.694135    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:04.795157 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.795116    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:04.963592 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:04.963564    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"]
Sep 08 14:53:04.965743 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:53:04.965714    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5039c9e0_3587_4249_bcaf_26fec07c8527.slice/crio-ba64ed6e84349da1be0ced7664c811c4d20aca15a46395afdfb66a089c959901 WatchSource:0}: Error finding container ba64ed6e84349da1be0ced7664c811c4d20aca15a46395afdfb66a089c959901: Status 404 returned error can't find the container with id ba64ed6e84349da1be0ced7664c811c4d20aca15a46395afdfb66a089c959901
Sep 08 14:53:05.296437 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:05.296404    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn" event={"ID":"5039c9e0-3587-4249-bcaf-26fec07c8527","Type":"ContainerStarted","Data":"9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e"}
Sep 08 14:53:05.296613 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:05.296445    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn" event={"ID":"5039c9e0-3587-4249-bcaf-26fec07c8527","Type":"ContainerStarted","Data":"ba64ed6e84349da1be0ced7664c811c4d20aca15a46395afdfb66a089c959901"}
Sep 08 14:53:05.322365 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:05.322316    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn" podStartSLOduration=1.322299664 podStartE2EDuration="1.322299664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:53:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:53:05.320495841 +0000 UTC m=+1136.048312857" watchObservedRunningTime="2026-09-08 14:53:05.322299664 +0000 UTC m=+1136.050116679"
Sep 08 14:53:14.686710 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:14.686666    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:14.686710 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:14.686719    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:14.699414 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:14.699384    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:15.358900 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:15.358874    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:53:31.200737 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:31.200658    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"]
Sep 08 14:53:31.204537 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:31.204521    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"
Sep 08 14:53:31.207393 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:31.207372    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:53:31.207620 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:31.207606    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 14:53:31.214869 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:31.214825    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"]
Sep 08 14:53:31.235741 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:31.235712    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/340354a0-f65d-4252-b512-4b896fb12e74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q\" (UID: \"340354a0-f65d-4252-b512-4b896fb12e74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"
Sep 08 14:53:31.337186 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:31.337148    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/340354a0-f65d-4252-b512-4b896fb12e74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q\" (UID: \"340354a0-f65d-4252-b512-4b896fb12e74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"
Sep 08 14:53:31.337782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:31.337760    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/340354a0-f65d-4252-b512-4b896fb12e74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q\" (UID: \"340354a0-f65d-4252-b512-4b896fb12e74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"
Sep 08 14:53:31.516899 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:31.516869    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"
Sep 08 14:53:31.792304 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:53:31.792274    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod340354a0_f65d_4252_b512_4b896fb12e74.slice/crio-73c89eb8e6f80593108c1f9d17c221e75c316a30e116a5bc9fb1cf81bc2caabd WatchSource:0}: Error finding container 73c89eb8e6f80593108c1f9d17c221e75c316a30e116a5bc9fb1cf81bc2caabd: Status 404 returned error can't find the container with id 73c89eb8e6f80593108c1f9d17c221e75c316a30e116a5bc9fb1cf81bc2caabd
Sep 08 14:53:31.793781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:31.793755    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"]
Sep 08 14:53:32.435429 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:32.435392    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q" event={"ID":"340354a0-f65d-4252-b512-4b896fb12e74","Type":"ContainerStarted","Data":"c715f01caa3fd780c9c5db37656edc3d83f4b36df63551971d7862c111520a59"}
Sep 08 14:53:32.435865 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:32.435443    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q" event={"ID":"340354a0-f65d-4252-b512-4b896fb12e74","Type":"ContainerStarted","Data":"73c89eb8e6f80593108c1f9d17c221e75c316a30e116a5bc9fb1cf81bc2caabd"}
Sep 08 14:53:32.455236 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:32.455189    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q" podStartSLOduration=1.4551748500000001 podStartE2EDuration="1.45517485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:53:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:53:32.453305079 +0000 UTC m=+1163.181122093" watchObservedRunningTime="2026-09-08 14:53:32.45517485 +0000 UTC m=+1163.182991892"
Sep 08 14:53:33.442494 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:33.442461    2583 generic.go:356] "Generic (PLEG): container finished" podID="340354a0-f65d-4252-b512-4b896fb12e74" containerID="c715f01caa3fd780c9c5db37656edc3d83f4b36df63551971d7862c111520a59" exitCode=0
Sep 08 14:53:33.442964 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:33.442560    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q" event={"ID":"340354a0-f65d-4252-b512-4b896fb12e74","Type":"ContainerDied","Data":"c715f01caa3fd780c9c5db37656edc3d83f4b36df63551971d7862c111520a59"}
Sep 08 14:53:34.569776 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:34.569753    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"
Sep 08 14:53:34.667665 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:34.667633    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/340354a0-f65d-4252-b512-4b896fb12e74-maas-api-service-ca\") pod \"340354a0-f65d-4252-b512-4b896fb12e74\" (UID: \"340354a0-f65d-4252-b512-4b896fb12e74\") "
Sep 08 14:53:34.667974 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:34.667953    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/340354a0-f65d-4252-b512-4b896fb12e74-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "340354a0-f65d-4252-b512-4b896fb12e74" (UID: "340354a0-f65d-4252-b512-4b896fb12e74"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:53:34.768849 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:34.768799    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/340354a0-f65d-4252-b512-4b896fb12e74-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:53:35.453417 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:35.453383    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q" event={"ID":"340354a0-f65d-4252-b512-4b896fb12e74","Type":"ContainerDied","Data":"73c89eb8e6f80593108c1f9d17c221e75c316a30e116a5bc9fb1cf81bc2caabd"}
Sep 08 14:53:35.453417 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:35.453402    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"
Sep 08 14:53:35.453624 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:35.453427    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73c89eb8e6f80593108c1f9d17c221e75c316a30e116a5bc9fb1cf81bc2caabd"
Sep 08 14:53:41.912088 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:41.912055    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:46.660772 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:46.660739    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:46.694293 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:46.694261    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"]
Sep 08 14:53:46.694564 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:46.694536    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss" podUID="155572f0-9d1d-4724-a282-e63fb60c9472" containerName="payload-pre-processing" containerID="cri-o://cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3" gracePeriod=30
Sep 08 14:53:46.761247 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:46.761215    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:46.860346 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:46.860314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:46.935529 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:46.935496    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"]
Sep 08 14:53:46.938794 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:46.938760    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:46.939583 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:46.939563    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3bb80f8b-vjr4q"]
Sep 08 14:53:46.957047 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:46.957027    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:53:47.081509 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.081478    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/155572f0-9d1d-4724-a282-e63fb60c9472-plugins-config-volume\") pod \"155572f0-9d1d-4724-a282-e63fb60c9472\" (UID: \"155572f0-9d1d-4724-a282-e63fb60c9472\") "
Sep 08 14:53:47.081647 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.081616    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgd6z\" (UniqueName: \"kubernetes.io/projected/155572f0-9d1d-4724-a282-e63fb60c9472-kube-api-access-dgd6z\") pod \"155572f0-9d1d-4724-a282-e63fb60c9472\" (UID: \"155572f0-9d1d-4724-a282-e63fb60c9472\") "
Sep 08 14:53:47.081951 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.081929    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/155572f0-9d1d-4724-a282-e63fb60c9472-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "155572f0-9d1d-4724-a282-e63fb60c9472" (UID: "155572f0-9d1d-4724-a282-e63fb60c9472"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:53:47.083728 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.083709    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/155572f0-9d1d-4724-a282-e63fb60c9472-kube-api-access-dgd6z" (OuterVolumeSpecName: "kube-api-access-dgd6z") pod "155572f0-9d1d-4724-a282-e63fb60c9472" (UID: "155572f0-9d1d-4724-a282-e63fb60c9472"). InnerVolumeSpecName "kube-api-access-dgd6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:53:47.183121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.183042    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgd6z\" (UniqueName: \"kubernetes.io/projected/155572f0-9d1d-4724-a282-e63fb60c9472-kube-api-access-dgd6z\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:53:47.183121 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.183080    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/155572f0-9d1d-4724-a282-e63fb60c9472-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:53:47.514583 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.514547    2583 generic.go:356] "Generic (PLEG): container finished" podID="155572f0-9d1d-4724-a282-e63fb60c9472" containerID="cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3" exitCode=0
Sep 08 14:53:47.514778 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.514600    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"
Sep 08 14:53:47.514778 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.514641    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss" event={"ID":"155572f0-9d1d-4724-a282-e63fb60c9472","Type":"ContainerDied","Data":"cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3"}
Sep 08 14:53:47.514778 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.514704    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss" event={"ID":"155572f0-9d1d-4724-a282-e63fb60c9472","Type":"ContainerDied","Data":"f97b9c67c9467c113eb80a3c352b0d9c3b29829218fd1d66187e08af17235d3c"}
Sep 08 14:53:47.514778 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.514735    2583 scope.go:117] "RemoveContainer" containerID="cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3"
Sep 08 14:53:47.523998 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.523981    2583 scope.go:117] "RemoveContainer" containerID="cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3"
Sep 08 14:53:47.524255 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:53:47.524234    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3\": container with ID starting with cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3 not found: ID does not exist" containerID="cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3"
Sep 08 14:53:47.524312 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.524265    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3"} err="failed to get container status \"cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3\": rpc error: code = NotFound desc = could not find container \"cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3\": container with ID starting with cf6ddca999eccd922638f87a298077efa308868ad3ed4246b3accb75ec9f85a3 not found: ID does not exist"
Sep 08 14:53:47.536653 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.536633    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"]
Sep 08 14:53:47.540453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.540434    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3bb80f8b-688c79694-fxzss"]
Sep 08 14:53:47.793497 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.793420    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="155572f0-9d1d-4724-a282-e63fb60c9472" path="/var/lib/kubelet/pods/155572f0-9d1d-4724-a282-e63fb60c9472/volumes"
Sep 08 14:53:47.793858 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:47.793732    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="340354a0-f65d-4252-b512-4b896fb12e74" path="/var/lib/kubelet/pods/340354a0-f65d-4252-b512-4b896fb12e74/volumes"
Sep 08 14:53:49.227258 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:49.227225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:49.994635 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:49.994603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:53:54.993349 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:54.993313    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"]
Sep 08 14:53:55.072888 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:53:55.072851    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:09.872381 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:09.872352    2583 scope.go:117] "RemoveContainer" containerID="2764574200b764ebcfaf41cd32922f880113e2838ad3f2e2681c03e425597c42"
Sep 08 14:54:10.014166 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.014101    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn" podUID="5039c9e0-3587-4249-bcaf-26fec07c8527" containerName="main" containerID="cri-o://9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e" gracePeriod=45
Sep 08 14:54:10.268451 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.268426    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:54:10.297978 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.297944    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5039c9e0-3587-4249-bcaf-26fec07c8527-tls-certs\") pod \"5039c9e0-3587-4249-bcaf-26fec07c8527\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") "
Sep 08 14:54:10.298160 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.298006    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-model-cache\") pod \"5039c9e0-3587-4249-bcaf-26fec07c8527\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") "
Sep 08 14:54:10.298160 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.298041    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-dshm\") pod \"5039c9e0-3587-4249-bcaf-26fec07c8527\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") "
Sep 08 14:54:10.298160 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.298068    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-tmp-dir\") pod \"5039c9e0-3587-4249-bcaf-26fec07c8527\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") "
Sep 08 14:54:10.298160 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.298109    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-home\") pod \"5039c9e0-3587-4249-bcaf-26fec07c8527\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") "
Sep 08 14:54:10.298160 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.298159    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28767\" (UniqueName: \"kubernetes.io/projected/5039c9e0-3587-4249-bcaf-26fec07c8527-kube-api-access-28767\") pod \"5039c9e0-3587-4249-bcaf-26fec07c8527\" (UID: \"5039c9e0-3587-4249-bcaf-26fec07c8527\") "
Sep 08 14:54:10.298448 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.298300    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-model-cache" (OuterVolumeSpecName: "model-cache") pod "5039c9e0-3587-4249-bcaf-26fec07c8527" (UID: "5039c9e0-3587-4249-bcaf-26fec07c8527"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:54:10.298560 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.298524    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-home" (OuterVolumeSpecName: "home") pod "5039c9e0-3587-4249-bcaf-26fec07c8527" (UID: "5039c9e0-3587-4249-bcaf-26fec07c8527"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:54:10.298633 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.298595    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5039c9e0-3587-4249-bcaf-26fec07c8527" (UID: "5039c9e0-3587-4249-bcaf-26fec07c8527"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:54:10.298633 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.298609    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:54:10.300363 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.300332    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-dshm" (OuterVolumeSpecName: "dshm") pod "5039c9e0-3587-4249-bcaf-26fec07c8527" (UID: "5039c9e0-3587-4249-bcaf-26fec07c8527"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:54:10.300475 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.300453    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5039c9e0-3587-4249-bcaf-26fec07c8527-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5039c9e0-3587-4249-bcaf-26fec07c8527" (UID: "5039c9e0-3587-4249-bcaf-26fec07c8527"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:54:10.300816 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.300797    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5039c9e0-3587-4249-bcaf-26fec07c8527-kube-api-access-28767" (OuterVolumeSpecName: "kube-api-access-28767") pod "5039c9e0-3587-4249-bcaf-26fec07c8527" (UID: "5039c9e0-3587-4249-bcaf-26fec07c8527"). InnerVolumeSpecName "kube-api-access-28767". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:54:10.400180 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.400090    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5039c9e0-3587-4249-bcaf-26fec07c8527-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:54:10.400180 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.400118    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:54:10.400180 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.400130    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:54:10.400180 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.400140    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5039c9e0-3587-4249-bcaf-26fec07c8527-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:54:10.400180 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.400151    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28767\" (UniqueName: \"kubernetes.io/projected/5039c9e0-3587-4249-bcaf-26fec07c8527-kube-api-access-28767\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:54:10.632501 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.632466    2583 generic.go:356] "Generic (PLEG): container finished" podID="5039c9e0-3587-4249-bcaf-26fec07c8527" containerID="9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e" exitCode=0
Sep 08 14:54:10.632703 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.632507    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"
Sep 08 14:54:10.632703 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.632557    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn" event={"ID":"5039c9e0-3587-4249-bcaf-26fec07c8527","Type":"ContainerDied","Data":"9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e"}
Sep 08 14:54:10.632703 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.632601    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn" event={"ID":"5039c9e0-3587-4249-bcaf-26fec07c8527","Type":"ContainerDied","Data":"ba64ed6e84349da1be0ced7664c811c4d20aca15a46395afdfb66a089c959901"}
Sep 08 14:54:10.632703 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.632626    2583 scope.go:117] "RemoveContainer" containerID="9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e"
Sep 08 14:54:10.642263 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.642244    2583 scope.go:117] "RemoveContainer" containerID="9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e"
Sep 08 14:54:10.642534 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:54:10.642513    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e\": container with ID starting with 9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e not found: ID does not exist" containerID="9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e"
Sep 08 14:54:10.642582 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.642544    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e"} err="failed to get container status \"9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e\": rpc error: code = NotFound desc = could not find container \"9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e\": container with ID starting with 9bb27889f8ae87e3541161a0ce389e6b26ea011d923e76cdb1807ff314643c4e not found: ID does not exist"
Sep 08 14:54:10.654332 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.654277    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"]
Sep 08 14:54:10.658805 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:10.658780    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3bb80f8b/e2e-lifecycle-model-3bb80f8b-kserve-d495cfbdc-6xhhn"]
Sep 08 14:54:11.793150 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:11.793108    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5039c9e0-3587-4249-bcaf-26fec07c8527" path="/var/lib/kubelet/pods/5039c9e0-3587-4249-bcaf-26fec07c8527/volumes"
Sep 08 14:54:31.267687 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:31.267657    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:32.142533 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:32.142491    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:37.837665 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:37.837630    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:37.936953 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:37.936903    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:38.086745 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:38.086715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:39.320982 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:39.320946    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:39.416839 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:39.416802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:39.516481 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:39.516448    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:40.147671 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:40.147638    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:40.251151 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:40.251119    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:41.595095 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:41.595065    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:47.454598 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:47.454557    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:49.260911 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:49.260867    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:50.917540 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:50.917505    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"]
Sep 08 14:54:50.928906 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:50.928850    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:51.874135 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:51.874099    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"]
Sep 08 14:54:51.898952 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:51.898916    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:54.160645 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:54.160614    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:54.266896 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:54.266861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:54:54.361348 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:54:54.361314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:04.836081 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:04.836006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:05.065980 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:05.065948    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:05.938530 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:05.938486    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm" podUID="f15ecc19-37d0-4591-8f68-5252cf61ab45" containerName="main" containerID="cri-o://057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150" gracePeriod=45
Sep 08 14:55:06.197440 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.197386    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:55:06.317312 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317284    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-home\") pod \"f15ecc19-37d0-4591-8f68-5252cf61ab45\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") "
Sep 08 14:55:06.317481 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317327    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-dshm\") pod \"f15ecc19-37d0-4591-8f68-5252cf61ab45\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") "
Sep 08 14:55:06.317481 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317349    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxmpf\" (UniqueName: \"kubernetes.io/projected/f15ecc19-37d0-4591-8f68-5252cf61ab45-kube-api-access-hxmpf\") pod \"f15ecc19-37d0-4591-8f68-5252cf61ab45\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") "
Sep 08 14:55:06.317481 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317383    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-tmp-dir\") pod \"f15ecc19-37d0-4591-8f68-5252cf61ab45\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") "
Sep 08 14:55:06.317481 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317428    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-model-cache\") pod \"f15ecc19-37d0-4591-8f68-5252cf61ab45\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") "
Sep 08 14:55:06.317481 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317453    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15ecc19-37d0-4591-8f68-5252cf61ab45-tls-certs\") pod \"f15ecc19-37d0-4591-8f68-5252cf61ab45\" (UID: \"f15ecc19-37d0-4591-8f68-5252cf61ab45\") "
Sep 08 14:55:06.317740 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317591    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-home" (OuterVolumeSpecName: "home") pod "f15ecc19-37d0-4591-8f68-5252cf61ab45" (UID: "f15ecc19-37d0-4591-8f68-5252cf61ab45"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:55:06.317740 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317669    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f15ecc19-37d0-4591-8f68-5252cf61ab45" (UID: "f15ecc19-37d0-4591-8f68-5252cf61ab45"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:55:06.317740 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317684    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-model-cache" (OuterVolumeSpecName: "model-cache") pod "f15ecc19-37d0-4591-8f68-5252cf61ab45" (UID: "f15ecc19-37d0-4591-8f68-5252cf61ab45"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:55:06.317929 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317770    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:06.317929 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317790    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:06.317929 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.317803    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:06.319689 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.319664    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-dshm" (OuterVolumeSpecName: "dshm") pod "f15ecc19-37d0-4591-8f68-5252cf61ab45" (UID: "f15ecc19-37d0-4591-8f68-5252cf61ab45"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:55:06.319797 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.319708    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15ecc19-37d0-4591-8f68-5252cf61ab45-kube-api-access-hxmpf" (OuterVolumeSpecName: "kube-api-access-hxmpf") pod "f15ecc19-37d0-4591-8f68-5252cf61ab45" (UID: "f15ecc19-37d0-4591-8f68-5252cf61ab45"). InnerVolumeSpecName "kube-api-access-hxmpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:55:06.319797 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.319728    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f15ecc19-37d0-4591-8f68-5252cf61ab45-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f15ecc19-37d0-4591-8f68-5252cf61ab45" (UID: "f15ecc19-37d0-4591-8f68-5252cf61ab45"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:55:06.418398 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.418367    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15ecc19-37d0-4591-8f68-5252cf61ab45-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:06.418398 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.418391    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15ecc19-37d0-4591-8f68-5252cf61ab45-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:06.418398 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.418402    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxmpf\" (UniqueName: \"kubernetes.io/projected/f15ecc19-37d0-4591-8f68-5252cf61ab45-kube-api-access-hxmpf\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:06.894476 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.894439    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z" podUID="9a687bf9-2a16-4bb7-9f22-a254d183561d" containerName="main" containerID="cri-o://55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7" gracePeriod=45
Sep 08 14:55:06.921759 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.921728    2583 generic.go:356] "Generic (PLEG): container finished" podID="f15ecc19-37d0-4591-8f68-5252cf61ab45" containerID="057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150" exitCode=0
Sep 08 14:55:06.921932 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.921773    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"
Sep 08 14:55:06.921932 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.921801    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm" event={"ID":"f15ecc19-37d0-4591-8f68-5252cf61ab45","Type":"ContainerDied","Data":"057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150"}
Sep 08 14:55:06.921932 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.921857    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm" event={"ID":"f15ecc19-37d0-4591-8f68-5252cf61ab45","Type":"ContainerDied","Data":"1fb607640d50d1eb98525dd59412d1d2cfa7c836cb7c9356efaa211cfc1ac25a"}
Sep 08 14:55:06.921932 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.921875    2583 scope.go:117] "RemoveContainer" containerID="057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150"
Sep 08 14:55:06.991200 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.991173    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"]
Sep 08 14:55:06.995581 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:06.995557    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/auth-test-model-ef8f61-kserve-5b468d654c-v9dtm"]
Sep 08 14:55:07.007965 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.007697    2583 scope.go:117] "RemoveContainer" containerID="057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150"
Sep 08 14:55:07.008194 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:55:07.008174    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150\": container with ID starting with 057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150 not found: ID does not exist" containerID="057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150"
Sep 08 14:55:07.008243 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.008203    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150"} err="failed to get container status \"057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150\": rpc error: code = NotFound desc = could not find container \"057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150\": container with ID starting with 057ec71b4392653ce639d336ac347465b6e77e220e15d5a67e97e665ccc7c150 not found: ID does not exist"
Sep 08 14:55:07.074256 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074225    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"]
Sep 08 14:55:07.074629 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074615    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="155572f0-9d1d-4724-a282-e63fb60c9472" containerName="payload-pre-processing"
Sep 08 14:55:07.074629 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074629    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="155572f0-9d1d-4724-a282-e63fb60c9472" containerName="payload-pre-processing"
Sep 08 14:55:07.074752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074643    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f15ecc19-37d0-4591-8f68-5252cf61ab45" containerName="main"
Sep 08 14:55:07.074752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074648    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f15ecc19-37d0-4591-8f68-5252cf61ab45" containerName="main"
Sep 08 14:55:07.074752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074658    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="340354a0-f65d-4252-b512-4b896fb12e74" containerName="revoke-keys"
Sep 08 14:55:07.074752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074665    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="340354a0-f65d-4252-b512-4b896fb12e74" containerName="revoke-keys"
Sep 08 14:55:07.074752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074674    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5039c9e0-3587-4249-bcaf-26fec07c8527" containerName="main"
Sep 08 14:55:07.074752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074680    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5039c9e0-3587-4249-bcaf-26fec07c8527" containerName="main"
Sep 08 14:55:07.074752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074732    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5039c9e0-3587-4249-bcaf-26fec07c8527" containerName="main"
Sep 08 14:55:07.074752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074740    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f15ecc19-37d0-4591-8f68-5252cf61ab45" containerName="main"
Sep 08 14:55:07.074752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074746    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="340354a0-f65d-4252-b512-4b896fb12e74" containerName="revoke-keys"
Sep 08 14:55:07.074752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.074752    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="155572f0-9d1d-4724-a282-e63fb60c9472" containerName="payload-pre-processing"
Sep 08 14:55:07.078050 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.078031    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.081266 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.081134    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ef8f61\"/\"default-dockercfg-ffnc6\""
Sep 08 14:55:07.081266 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.081134    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ef8f61\"/\"sub-test-model-ef8f61-kserve-self-signed-certs\""
Sep 08 14:55:07.081452 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.081396    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ef8f61\"/\"kube-root-ca.crt\""
Sep 08 14:55:07.081501 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.081476    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ef8f61\"/\"openshift-service-ca.crt\""
Sep 08 14:55:07.089683 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.089659    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"]
Sep 08 14:55:07.100579 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.100554    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:07.157321 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.157268    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:55:07.226947 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.226921    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-dshm\") pod \"9a687bf9-2a16-4bb7-9f22-a254d183561d\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") "
Sep 08 14:55:07.227129 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.226967    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a687bf9-2a16-4bb7-9f22-a254d183561d-tls-certs\") pod \"9a687bf9-2a16-4bb7-9f22-a254d183561d\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") "
Sep 08 14:55:07.227129 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.226985    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-model-cache\") pod \"9a687bf9-2a16-4bb7-9f22-a254d183561d\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") "
Sep 08 14:55:07.227129 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227020    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-tmp-dir\") pod \"9a687bf9-2a16-4bb7-9f22-a254d183561d\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") "
Sep 08 14:55:07.227129 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227058    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9khgp\" (UniqueName: \"kubernetes.io/projected/9a687bf9-2a16-4bb7-9f22-a254d183561d-kube-api-access-9khgp\") pod \"9a687bf9-2a16-4bb7-9f22-a254d183561d\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") "
Sep 08 14:55:07.227424 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227198    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-home\") pod \"9a687bf9-2a16-4bb7-9f22-a254d183561d\" (UID: \"9a687bf9-2a16-4bb7-9f22-a254d183561d\") "
Sep 08 14:55:07.227424 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227328    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-model-cache" (OuterVolumeSpecName: "model-cache") pod "9a687bf9-2a16-4bb7-9f22-a254d183561d" (UID: "9a687bf9-2a16-4bb7-9f22-a254d183561d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:55:07.227424 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227339    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d64fbb59-30c4-466e-bc27-a98234bffee4-tls-certs\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.227424 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227342    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9a687bf9-2a16-4bb7-9f22-a254d183561d" (UID: "9a687bf9-2a16-4bb7-9f22-a254d183561d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:55:07.227628 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227454    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-tmp-dir\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.227628 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227474    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-home" (OuterVolumeSpecName: "home") pod "9a687bf9-2a16-4bb7-9f22-a254d183561d" (UID: "9a687bf9-2a16-4bb7-9f22-a254d183561d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:55:07.227628 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227489    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlrqk\" (UniqueName: \"kubernetes.io/projected/d64fbb59-30c4-466e-bc27-a98234bffee4-kube-api-access-jlrqk\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.227628 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-model-cache\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.227628 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227618    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-dshm\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.228087 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227646    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-home\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.228087 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227820    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:07.228087 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227864    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:07.228087 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.227881    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:07.229402 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.229379    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-dshm" (OuterVolumeSpecName: "dshm") pod "9a687bf9-2a16-4bb7-9f22-a254d183561d" (UID: "9a687bf9-2a16-4bb7-9f22-a254d183561d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:55:07.229505 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.229431    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a687bf9-2a16-4bb7-9f22-a254d183561d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9a687bf9-2a16-4bb7-9f22-a254d183561d" (UID: "9a687bf9-2a16-4bb7-9f22-a254d183561d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:55:07.229505 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.229488    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a687bf9-2a16-4bb7-9f22-a254d183561d-kube-api-access-9khgp" (OuterVolumeSpecName: "kube-api-access-9khgp") pod "9a687bf9-2a16-4bb7-9f22-a254d183561d" (UID: "9a687bf9-2a16-4bb7-9f22-a254d183561d"). InnerVolumeSpecName "kube-api-access-9khgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:55:07.329147 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329114    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-dshm\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.329147 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329151    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-home\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.329373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329172    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d64fbb59-30c4-466e-bc27-a98234bffee4-tls-certs\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.329373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329206    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-tmp-dir\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.329373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329230    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlrqk\" (UniqueName: \"kubernetes.io/projected/d64fbb59-30c4-466e-bc27-a98234bffee4-kube-api-access-jlrqk\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.329373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329253    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-model-cache\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.329373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329303    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9khgp\" (UniqueName: \"kubernetes.io/projected/9a687bf9-2a16-4bb7-9f22-a254d183561d-kube-api-access-9khgp\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:07.329373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329320    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a687bf9-2a16-4bb7-9f22-a254d183561d-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:07.329373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329335    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a687bf9-2a16-4bb7-9f22-a254d183561d-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:55:07.329689 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329590    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-home\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.329689 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329630    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-tmp-dir\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.329689 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.329678    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-model-cache\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.331461 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.331440    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-dshm\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.331776 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.331753    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d64fbb59-30c4-466e-bc27-a98234bffee4-tls-certs\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.339039 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.339022    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlrqk\" (UniqueName: \"kubernetes.io/projected/d64fbb59-30c4-466e-bc27-a98234bffee4-kube-api-access-jlrqk\") pod \"sub-test-model-ef8f61-kserve-787f44787f-flr4r\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.394608 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.394580    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:07.670286 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.670255    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"]
Sep 08 14:55:07.673933 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:55:07.673902    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd64fbb59_30c4_466e_bc27_a98234bffee4.slice/crio-2f164b6927103275e044c72bf119dba416a464ec148aa5fa443241ea7a17baec WatchSource:0}: Error finding container 2f164b6927103275e044c72bf119dba416a464ec148aa5fa443241ea7a17baec: Status 404 returned error can't find the container with id 2f164b6927103275e044c72bf119dba416a464ec148aa5fa443241ea7a17baec
Sep 08 14:55:07.793933 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.793901    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f15ecc19-37d0-4591-8f68-5252cf61ab45" path="/var/lib/kubelet/pods/f15ecc19-37d0-4591-8f68-5252cf61ab45/volumes"
Sep 08 14:55:07.929304 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.929269    2583 generic.go:356] "Generic (PLEG): container finished" podID="9a687bf9-2a16-4bb7-9f22-a254d183561d" containerID="55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7" exitCode=0
Sep 08 14:55:07.929492 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.929312    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"
Sep 08 14:55:07.929492 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.929323    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z" event={"ID":"9a687bf9-2a16-4bb7-9f22-a254d183561d","Type":"ContainerDied","Data":"55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7"}
Sep 08 14:55:07.929492 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.929381    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z" event={"ID":"9a687bf9-2a16-4bb7-9f22-a254d183561d","Type":"ContainerDied","Data":"7db001eef8430d0e090812bce11abef10effd9a60c94ab478692e78e1a135860"}
Sep 08 14:55:07.929492 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.929407    2583 scope.go:117] "RemoveContainer" containerID="55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7"
Sep 08 14:55:07.932753 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.932725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r" event={"ID":"d64fbb59-30c4-466e-bc27-a98234bffee4","Type":"ContainerStarted","Data":"6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d"}
Sep 08 14:55:07.932926 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.932767    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r" event={"ID":"d64fbb59-30c4-466e-bc27-a98234bffee4","Type":"ContainerStarted","Data":"2f164b6927103275e044c72bf119dba416a464ec148aa5fa443241ea7a17baec"}
Sep 08 14:55:07.938715 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.938692    2583 scope.go:117] "RemoveContainer" containerID="55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7"
Sep 08 14:55:07.939075 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:55:07.939052    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7\": container with ID starting with 55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7 not found: ID does not exist" containerID="55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7"
Sep 08 14:55:07.939151 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.939084    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7"} err="failed to get container status \"55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7\": rpc error: code = NotFound desc = could not find container \"55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7\": container with ID starting with 55e3d1674dd8ccc1a519ff2969be036d220cffa3e3df21569298ef685714cba7 not found: ID does not exist"
Sep 08 14:55:07.953611 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.953563    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r" podStartSLOduration=0.95354723 podStartE2EDuration="953.54723ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:55:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:55:07.95150574 +0000 UTC m=+1258.679322754" watchObservedRunningTime="2026-09-08 14:55:07.95354723 +0000 UTC m=+1258.681364245"
Sep 08 14:55:07.968138 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.968077    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"]
Sep 08 14:55:07.971243 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:07.971216    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/auth-test-model-f83533-kserve-77789dbc55-whp8z"]
Sep 08 14:55:09.794010 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:09.793980    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a687bf9-2a16-4bb7-9f22-a254d183561d" path="/var/lib/kubelet/pods/9a687bf9-2a16-4bb7-9f22-a254d183561d/volumes"
Sep 08 14:55:09.897530 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:09.897499    2583 scope.go:117] "RemoveContainer" containerID="ca274711a7600b785e12d135e8780f5fffdd34dfecba84efc60a50cad28d7573"
Sep 08 14:55:17.394966 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:17.394933    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:17.395546 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:17.395082    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:17.407902 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:17.407877    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:17.999323 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:17.999284    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:55:25.965275 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:25.965225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:31.895161 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.895131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:31.902201 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.902170    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"]
Sep 08 14:55:31.902559 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.902548    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a687bf9-2a16-4bb7-9f22-a254d183561d" containerName="main"
Sep 08 14:55:31.902603 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.902562    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a687bf9-2a16-4bb7-9f22-a254d183561d" containerName="main"
Sep 08 14:55:31.902639 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.902621    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a687bf9-2a16-4bb7-9f22-a254d183561d" containerName="main"
Sep 08 14:55:31.907404 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.907384    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:31.910593 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.910573    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"kube-root-ca.crt\""
Sep 08 14:55:31.911670 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.911651    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"openshift-service-ca.crt\""
Sep 08 14:55:31.911790 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.911651    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"default-dockercfg-mxzdm\""
Sep 08 14:55:31.911790 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.911686    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"sub-test-model-f83533-kserve-self-signed-certs\""
Sep 08 14:55:31.921754 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.921729    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"]
Sep 08 14:55:31.955076 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.955050    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-home\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:31.955239 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.955105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-tmp-dir\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:31.955239 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.955126    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-dshm\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:31.955239 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.955189    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-model-cache\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:31.955239 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.955215    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46dfc811-340c-4bc9-a07a-f31ca0a27931-tls-certs\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:31.955407 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.955264    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68mc8\" (UniqueName: \"kubernetes.io/projected/46dfc811-340c-4bc9-a07a-f31ca0a27931-kube-api-access-68mc8\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:31.982984 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:31.982956    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:32.055797 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.055768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-model-cache\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.055797 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.055800    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46dfc811-340c-4bc9-a07a-f31ca0a27931-tls-certs\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.056044 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.055872    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68mc8\" (UniqueName: \"kubernetes.io/projected/46dfc811-340c-4bc9-a07a-f31ca0a27931-kube-api-access-68mc8\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.056044 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.055924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-home\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.056044 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.055983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-tmp-dir\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.056044 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.056001    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-dshm\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.056283 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.056260    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-model-cache\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.056370 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.056316    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-home\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.056424 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.056369    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-tmp-dir\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.058350 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.058325    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-dshm\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.058534 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.058517    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46dfc811-340c-4bc9-a07a-f31ca0a27931-tls-certs\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.064998 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.064975    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68mc8\" (UniqueName: \"kubernetes.io/projected/46dfc811-340c-4bc9-a07a-f31ca0a27931-kube-api-access-68mc8\") pod \"sub-test-model-f83533-kserve-7597dc7d5f-7rj9r\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") " pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.217335 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.217250    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:32.235329 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.235300    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:32.496498 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.496464    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"]
Sep 08 14:55:32.499878 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:55:32.499847    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46dfc811_340c_4bc9_a07a_f31ca0a27931.slice/crio-d89f3b997c8e7da924fb7d4c169abd4ef991c4cf5fdd131c03721c9c5a1c2555 WatchSource:0}: Error finding container d89f3b997c8e7da924fb7d4c169abd4ef991c4cf5fdd131c03721c9c5a1c2555: Status 404 returned error can't find the container with id d89f3b997c8e7da924fb7d4c169abd4ef991c4cf5fdd131c03721c9c5a1c2555
Sep 08 14:55:32.706516 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:32.706483    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:33.005294 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:33.005264    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:33.072098 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:33.072070    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r" event={"ID":"46dfc811-340c-4bc9-a07a-f31ca0a27931","Type":"ContainerStarted","Data":"3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a"}
Sep 08 14:55:33.072250 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:33.072109    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r" event={"ID":"46dfc811-340c-4bc9-a07a-f31ca0a27931","Type":"ContainerStarted","Data":"d89f3b997c8e7da924fb7d4c169abd4ef991c4cf5fdd131c03721c9c5a1c2555"}
Sep 08 14:55:33.090029 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:33.089984    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r" podStartSLOduration=2.089969901 podStartE2EDuration="2.089969901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:55:33.089062398 +0000 UTC m=+1283.816879412" watchObservedRunningTime="2026-09-08 14:55:33.089969901 +0000 UTC m=+1283.817786915"
Sep 08 14:55:35.699933 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:35.699897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:37.399044 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.399006    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"]
Sep 08 14:55:37.402491 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.402468    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:55:37.405794 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.405766    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-1145ca-dockercfg-zpfvq\""
Sep 08 14:55:37.405955 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.405938    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-1145ca\""
Sep 08 14:55:37.412400 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.412373    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"]
Sep 08 14:55:37.441493 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.441467    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"]
Sep 08 14:55:37.444895 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.444866    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:55:37.455884 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.455862    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"]
Sep 08 14:55:37.506070 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.506039    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513df6e4-de9b-4481-a0e8-33be4a6a034c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch\" (UID: \"513df6e4-de9b-4481-a0e8-33be4a6a034c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:55:37.506233 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.506105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8qbl\" (UniqueName: \"kubernetes.io/projected/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-kube-api-access-m8qbl\") pod \"payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7\" (UID: \"d4490b6d-08ad-40ce-ab85-d8c8f70645a5\") " pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:55:37.506233 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.506192    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7ksk\" (UniqueName: \"kubernetes.io/projected/513df6e4-de9b-4481-a0e8-33be4a6a034c-kube-api-access-g7ksk\") pod \"payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch\" (UID: \"513df6e4-de9b-4481-a0e8-33be4a6a034c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:55:37.506320 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.506251    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7\" (UID: \"d4490b6d-08ad-40ce-ab85-d8c8f70645a5\") " pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:55:37.607356 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.607329    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513df6e4-de9b-4481-a0e8-33be4a6a034c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch\" (UID: \"513df6e4-de9b-4481-a0e8-33be4a6a034c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:55:37.607499 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.607381    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8qbl\" (UniqueName: \"kubernetes.io/projected/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-kube-api-access-m8qbl\") pod \"payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7\" (UID: \"d4490b6d-08ad-40ce-ab85-d8c8f70645a5\") " pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:55:37.607499 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.607422    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7ksk\" (UniqueName: \"kubernetes.io/projected/513df6e4-de9b-4481-a0e8-33be4a6a034c-kube-api-access-g7ksk\") pod \"payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch\" (UID: \"513df6e4-de9b-4481-a0e8-33be4a6a034c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:55:37.607579 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.607552    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7\" (UID: \"d4490b6d-08ad-40ce-ab85-d8c8f70645a5\") " pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:55:37.608077 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.608051    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513df6e4-de9b-4481-a0e8-33be4a6a034c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch\" (UID: \"513df6e4-de9b-4481-a0e8-33be4a6a034c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:55:37.608284 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.608262    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7\" (UID: \"d4490b6d-08ad-40ce-ab85-d8c8f70645a5\") " pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:55:37.616069 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.616049    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7ksk\" (UniqueName: \"kubernetes.io/projected/513df6e4-de9b-4481-a0e8-33be4a6a034c-kube-api-access-g7ksk\") pod \"payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch\" (UID: \"513df6e4-de9b-4481-a0e8-33be4a6a034c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:55:37.616069 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.616062    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8qbl\" (UniqueName: \"kubernetes.io/projected/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-kube-api-access-m8qbl\") pod \"payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7\" (UID: \"d4490b6d-08ad-40ce-ab85-d8c8f70645a5\") " pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:55:37.714498 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.714427    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:55:37.755311 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.755268    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:55:37.993257 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:37.992524    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"]
Sep 08 14:55:38.009852 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:38.008196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:38.060146 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:38.060118    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"]
Sep 08 14:55:38.101748 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:38.101483    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7" event={"ID":"d4490b6d-08ad-40ce-ab85-d8c8f70645a5","Type":"ContainerStarted","Data":"0e41e64c7c2830dac5c5e9acd5252a297ff3a5a371b49c1b96013a37eb5178a6"}
Sep 08 14:55:38.104460 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:38.103290    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch" event={"ID":"513df6e4-de9b-4481-a0e8-33be4a6a034c","Type":"ContainerStarted","Data":"dfe21a095fee55b247670e5b86ee99e0c11eee4ce479d8948da590f609b3d70d"}
Sep 08 14:55:38.106619 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:38.106592    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:55:39.111198 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:39.111165    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7" event={"ID":"d4490b6d-08ad-40ce-ab85-d8c8f70645a5","Type":"ContainerStarted","Data":"938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758"}
Sep 08 14:55:39.111670 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:39.111246    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:55:39.113034 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:39.113004    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch" event={"ID":"513df6e4-de9b-4481-a0e8-33be4a6a034c","Type":"ContainerStarted","Data":"5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896"}
Sep 08 14:55:39.113165 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:39.113143    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:55:39.131527 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:39.131480    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7" podStartSLOduration=2.131461509 podStartE2EDuration="2.131461509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:55:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:55:39.128773252 +0000 UTC m=+1289.856590268" watchObservedRunningTime="2026-09-08 14:55:39.131461509 +0000 UTC m=+1289.859278524"
Sep 08 14:55:39.146253 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:39.146197    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch" podStartSLOduration=2.146178698 podStartE2EDuration="2.146178698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:55:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:55:39.144411698 +0000 UTC m=+1289.872228714" watchObservedRunningTime="2026-09-08 14:55:39.146178698 +0000 UTC m=+1289.873995713"
Sep 08 14:55:42.217373 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:42.217341    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:42.217825 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:42.217405    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:42.230617 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:42.230595    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:43.148261 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:43.148237    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:55:50.124147 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:50.124112    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:55:50.124604 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:55:50.124210    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:56:00.582402 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:00.582367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:01.020794 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:01.020760    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:01.407235 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:01.407146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:01.504194 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:01.504159    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:03.983792 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:03.983758    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:04.081984 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:04.081945    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:05.115009 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.114972    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:05.215934 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.215900    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:05.502570 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.502532    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:05.697304 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.697274    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:05.787872 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.787759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:05.797117 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.797093    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"]
Sep 08 14:56:05.801605 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.801580    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:56:05.804252 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.804236    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-4e2141-dockercfg-stb68\""
Sep 08 14:56:05.804331 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.804296    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-4e2141\""
Sep 08 14:56:05.815161 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.815139    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"]
Sep 08 14:56:05.969125 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.969089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50c83458-3ac9-47bb-ad69-7a5838f3e069-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8\" (UID: \"50c83458-3ac9-47bb-ad69-7a5838f3e069\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:56:05.969125 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:05.969126    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k95ml\" (UniqueName: \"kubernetes.io/projected/50c83458-3ac9-47bb-ad69-7a5838f3e069-kube-api-access-k95ml\") pod \"payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8\" (UID: \"50c83458-3ac9-47bb-ad69-7a5838f3e069\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:56:06.063354 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:06.063250    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:06.070447 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:06.070426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50c83458-3ac9-47bb-ad69-7a5838f3e069-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8\" (UID: \"50c83458-3ac9-47bb-ad69-7a5838f3e069\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:56:06.070564 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:06.070454    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k95ml\" (UniqueName: \"kubernetes.io/projected/50c83458-3ac9-47bb-ad69-7a5838f3e069-kube-api-access-k95ml\") pod \"payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8\" (UID: \"50c83458-3ac9-47bb-ad69-7a5838f3e069\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:56:06.071199 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:06.071179    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50c83458-3ac9-47bb-ad69-7a5838f3e069-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8\" (UID: \"50c83458-3ac9-47bb-ad69-7a5838f3e069\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:56:06.086536 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:06.086508    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k95ml\" (UniqueName: \"kubernetes.io/projected/50c83458-3ac9-47bb-ad69-7a5838f3e069-kube-api-access-k95ml\") pod \"payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8\" (UID: \"50c83458-3ac9-47bb-ad69-7a5838f3e069\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:56:06.112494 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:06.112473    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:56:06.389134 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:06.389111    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"]
Sep 08 14:56:06.391535 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:56:06.391504    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c83458_3ac9_47bb_ad69_7a5838f3e069.slice/crio-3ffc53a373fb5b5b5396d51c3ec0c92b78fb38f65ead029a911993d92ed2df82 WatchSource:0}: Error finding container 3ffc53a373fb5b5b5396d51c3ec0c92b78fb38f65ead029a911993d92ed2df82: Status 404 returned error can't find the container with id 3ffc53a373fb5b5b5396d51c3ec0c92b78fb38f65ead029a911993d92ed2df82
Sep 08 14:56:06.806376 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:06.806343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:07.260436 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:07.260396    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8" event={"ID":"50c83458-3ac9-47bb-ad69-7a5838f3e069","Type":"ContainerStarted","Data":"892ec8ea9933eb6461aa68c03008dc012f1460b44679a3769b8683658248b4f4"}
Sep 08 14:56:07.260613 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:07.260462    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8" event={"ID":"50c83458-3ac9-47bb-ad69-7a5838f3e069","Type":"ContainerStarted","Data":"3ffc53a373fb5b5b5396d51c3ec0c92b78fb38f65ead029a911993d92ed2df82"}
Sep 08 14:56:07.260613 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:07.260581    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:56:07.279293 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:07.279242    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8" podStartSLOduration=2.279228772 podStartE2EDuration="2.279228772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:56:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:56:07.277776334 +0000 UTC m=+1318.005593352" watchObservedRunningTime="2026-09-08 14:56:07.279228772 +0000 UTC m=+1318.007045785"
Sep 08 14:56:18.267627 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:18.267596    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:56:27.151575 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:27.151539    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"]
Sep 08 14:56:27.185409 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:27.185372    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:28.081866 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:28.081810    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"]
Sep 08 14:56:28.119967 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:28.119932    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:28.883955 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:28.883916    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:31.655098 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:31.655023    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"]
Sep 08 14:56:31.659765 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:31.659749    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"
Sep 08 14:56:31.662680 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:31.662663    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:56:31.662945 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:31.662919    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 14:56:31.667003 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:31.666978    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"]
Sep 08 14:56:31.795411 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:31.795373    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:31.819925 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:31.819889    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be3e1d0-1b56-43ad-9dda-a320ccd20197-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1145ca-td62b\" (UID: \"0be3e1d0-1b56-43ad-9dda-a320ccd20197\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"
Sep 08 14:56:31.920998 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:31.920915    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be3e1d0-1b56-43ad-9dda-a320ccd20197-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1145ca-td62b\" (UID: \"0be3e1d0-1b56-43ad-9dda-a320ccd20197\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"
Sep 08 14:56:31.921537 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:31.921516    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be3e1d0-1b56-43ad-9dda-a320ccd20197-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1145ca-td62b\" (UID: \"0be3e1d0-1b56-43ad-9dda-a320ccd20197\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"
Sep 08 14:56:31.971588 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:31.971560    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"
Sep 08 14:56:32.244370 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:32.244344    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"]
Sep 08 14:56:32.246951 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:56:32.246920    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be3e1d0_1b56_43ad_9dda_a320ccd20197.slice/crio-8ffb9254b9c2731b0ea2a55f61eed91e9e618810ed4cd71801fc4b7e4b044a52 WatchSource:0}: Error finding container 8ffb9254b9c2731b0ea2a55f61eed91e9e618810ed4cd71801fc4b7e4b044a52: Status 404 returned error can't find the container with id 8ffb9254b9c2731b0ea2a55f61eed91e9e618810ed4cd71801fc4b7e4b044a52
Sep 08 14:56:32.394572 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:32.394539    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b" event={"ID":"0be3e1d0-1b56-43ad-9dda-a320ccd20197","Type":"ContainerStarted","Data":"8ffb9254b9c2731b0ea2a55f61eed91e9e618810ed4cd71801fc4b7e4b044a52"}
Sep 08 14:56:33.403093 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:33.403007    2583 generic.go:356] "Generic (PLEG): container finished" podID="0be3e1d0-1b56-43ad-9dda-a320ccd20197" containerID="d9348ffc26f6b1e97825a2a00c8957ca4065b0f88b4b9dac6e1d4caec59749a4" exitCode=0
Sep 08 14:56:33.403093 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:33.403069    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b" event={"ID":"0be3e1d0-1b56-43ad-9dda-a320ccd20197","Type":"ContainerDied","Data":"d9348ffc26f6b1e97825a2a00c8957ca4065b0f88b4b9dac6e1d4caec59749a4"}
Sep 08 14:56:33.865906 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:33.865874    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"]
Sep 08 14:56:33.869480 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:33.869458    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:33.871540 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:33.871516    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:33.872445 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:33.872425    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ef8f61\"/\"rate-test-model-ef8f61-kserve-self-signed-certs\""
Sep 08 14:56:33.878925 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:33.878892    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"]
Sep 08 14:56:34.042941 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.042907    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsvtm\" (UniqueName: \"kubernetes.io/projected/0aa948a8-d77a-4115-811e-25f2743607ef-kube-api-access-gsvtm\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.043082 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.042974    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-tmp-dir\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.043082 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.043025    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-model-cache\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.043082 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.043068    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa948a8-d77a-4115-811e-25f2743607ef-tls-certs\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.043259 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.043102    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-home\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.043259 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.043141    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-dshm\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.143973 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.143886    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-tmp-dir\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.143973 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.143928    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-model-cache\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.144192 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.144053    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa948a8-d77a-4115-811e-25f2743607ef-tls-certs\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.144192 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.144104    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-home\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.144192 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.144136    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-dshm\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.144348 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.144220    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsvtm\" (UniqueName: \"kubernetes.io/projected/0aa948a8-d77a-4115-811e-25f2743607ef-kube-api-access-gsvtm\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.144348 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.144252    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-tmp-dir\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.144433 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.144343    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-model-cache\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.144481 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.144459    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-home\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.146560 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.146531    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-dshm\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.146771 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.146752    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa948a8-d77a-4115-811e-25f2743607ef-tls-certs\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.152383 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.152364    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsvtm\" (UniqueName: \"kubernetes.io/projected/0aa948a8-d77a-4115-811e-25f2743607ef-kube-api-access-gsvtm\") pod \"rate-test-model-ef8f61-kserve-58c69d99d7-rnptq\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") " pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.179498 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.179472    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:34.205617 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.205590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:34.452175 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.452029    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"]
Sep 08 14:56:34.454537 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:56:34.454507    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aa948a8_d77a_4115_811e_25f2743607ef.slice/crio-c6ce7d3a4a40552cf70c2c7adf04b9eac8a6edf2340c5f35383feea7e5a9dd48 WatchSource:0}: Error finding container c6ce7d3a4a40552cf70c2c7adf04b9eac8a6edf2340c5f35383feea7e5a9dd48: Status 404 returned error can't find the container with id c6ce7d3a4a40552cf70c2c7adf04b9eac8a6edf2340c5f35383feea7e5a9dd48
Sep 08 14:56:34.535051 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.535029    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"
Sep 08 14:56:34.648681 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.648643    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be3e1d0-1b56-43ad-9dda-a320ccd20197-maas-api-service-ca\") pod \"0be3e1d0-1b56-43ad-9dda-a320ccd20197\" (UID: \"0be3e1d0-1b56-43ad-9dda-a320ccd20197\") "
Sep 08 14:56:34.649009 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.648987    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be3e1d0-1b56-43ad-9dda-a320ccd20197-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0be3e1d0-1b56-43ad-9dda-a320ccd20197" (UID: "0be3e1d0-1b56-43ad-9dda-a320ccd20197"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:56:34.749500 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:34.749475    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be3e1d0-1b56-43ad-9dda-a320ccd20197-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:35.415623 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:35.415530    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq" event={"ID":"0aa948a8-d77a-4115-811e-25f2743607ef","Type":"ContainerStarted","Data":"6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c"}
Sep 08 14:56:35.415623 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:35.415572    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq" event={"ID":"0aa948a8-d77a-4115-811e-25f2743607ef","Type":"ContainerStarted","Data":"c6ce7d3a4a40552cf70c2c7adf04b9eac8a6edf2340c5f35383feea7e5a9dd48"}
Sep 08 14:56:35.416775 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:35.416746    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"
Sep 08 14:56:35.416888 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:35.416792    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b" event={"ID":"0be3e1d0-1b56-43ad-9dda-a320ccd20197","Type":"ContainerDied","Data":"8ffb9254b9c2731b0ea2a55f61eed91e9e618810ed4cd71801fc4b7e4b044a52"}
Sep 08 14:56:35.416888 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:35.416851    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ffb9254b9c2731b0ea2a55f61eed91e9e618810ed4cd71801fc4b7e4b044a52"
Sep 08 14:56:35.433422 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:35.433379    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq" podStartSLOduration=2.43336582 podStartE2EDuration="2.43336582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:56:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:56:35.432434817 +0000 UTC m=+1346.160251832" watchObservedRunningTime="2026-09-08 14:56:35.43336582 +0000 UTC m=+1346.161182857"
Sep 08 14:56:41.791651 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:41.791575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:42.385179 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.385128    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r" podUID="d64fbb59-30c4-466e-bc27-a98234bffee4" containerName="main" containerID="cri-o://6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d" gracePeriod=45
Sep 08 14:56:42.636122 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.636060    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:56:42.723544 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.723518    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-model-cache\") pod \"d64fbb59-30c4-466e-bc27-a98234bffee4\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") "
Sep 08 14:56:42.723731 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.723581    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-dshm\") pod \"d64fbb59-30c4-466e-bc27-a98234bffee4\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") "
Sep 08 14:56:42.723731 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.723619    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlrqk\" (UniqueName: \"kubernetes.io/projected/d64fbb59-30c4-466e-bc27-a98234bffee4-kube-api-access-jlrqk\") pod \"d64fbb59-30c4-466e-bc27-a98234bffee4\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") "
Sep 08 14:56:42.723731 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.723643    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d64fbb59-30c4-466e-bc27-a98234bffee4-tls-certs\") pod \"d64fbb59-30c4-466e-bc27-a98234bffee4\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") "
Sep 08 14:56:42.723731 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.723675    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-tmp-dir\") pod \"d64fbb59-30c4-466e-bc27-a98234bffee4\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") "
Sep 08 14:56:42.723731 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.723707    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-home\") pod \"d64fbb59-30c4-466e-bc27-a98234bffee4\" (UID: \"d64fbb59-30c4-466e-bc27-a98234bffee4\") "
Sep 08 14:56:42.724094 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.723771    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-model-cache" (OuterVolumeSpecName: "model-cache") pod "d64fbb59-30c4-466e-bc27-a98234bffee4" (UID: "d64fbb59-30c4-466e-bc27-a98234bffee4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:56:42.724094 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.724011    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d64fbb59-30c4-466e-bc27-a98234bffee4" (UID: "d64fbb59-30c4-466e-bc27-a98234bffee4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:56:42.724094 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.724019    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-home" (OuterVolumeSpecName: "home") pod "d64fbb59-30c4-466e-bc27-a98234bffee4" (UID: "d64fbb59-30c4-466e-bc27-a98234bffee4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:56:42.724094 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.724066    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:42.726026 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.725988    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d64fbb59-30c4-466e-bc27-a98234bffee4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d64fbb59-30c4-466e-bc27-a98234bffee4" (UID: "d64fbb59-30c4-466e-bc27-a98234bffee4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:56:42.726150 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.726062    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-dshm" (OuterVolumeSpecName: "dshm") pod "d64fbb59-30c4-466e-bc27-a98234bffee4" (UID: "d64fbb59-30c4-466e-bc27-a98234bffee4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:56:42.726150 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.726090    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d64fbb59-30c4-466e-bc27-a98234bffee4-kube-api-access-jlrqk" (OuterVolumeSpecName: "kube-api-access-jlrqk") pod "d64fbb59-30c4-466e-bc27-a98234bffee4" (UID: "d64fbb59-30c4-466e-bc27-a98234bffee4"). InnerVolumeSpecName "kube-api-access-jlrqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:56:42.824561 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.824523    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlrqk\" (UniqueName: \"kubernetes.io/projected/d64fbb59-30c4-466e-bc27-a98234bffee4-kube-api-access-jlrqk\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:42.824561 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.824556    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d64fbb59-30c4-466e-bc27-a98234bffee4-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:42.824980 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.824572    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:42.824980 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.824583    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:42.824980 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:42.824594    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d64fbb59-30c4-466e-bc27-a98234bffee4-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:43.104867 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.104787    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r" podUID="46dfc811-340c-4bc9-a07a-f31ca0a27931" containerName="main" containerID="cri-o://3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a" gracePeriod=45
Sep 08 14:56:43.132867 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:56:43.132823    2583 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46dfc811_340c_4bc9_a07a_f31ca0a27931.slice/crio-3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a.scope\": RecentStats: unable to find data in memory cache]"
Sep 08 14:56:43.138049 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.138010    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r" podUID="46dfc811-340c-4bc9-a07a-f31ca0a27931" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": dial tcp 10.133.0.69:8000: connect: connection refused"
Sep 08 14:56:43.349315 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.349293    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:56:43.428639 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.428607    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-model-cache\") pod \"46dfc811-340c-4bc9-a07a-f31ca0a27931\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") "
Sep 08 14:56:43.428791 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.428652    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-dshm\") pod \"46dfc811-340c-4bc9-a07a-f31ca0a27931\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") "
Sep 08 14:56:43.428791 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.428670    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-home\") pod \"46dfc811-340c-4bc9-a07a-f31ca0a27931\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") "
Sep 08 14:56:43.428791 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.428686    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46dfc811-340c-4bc9-a07a-f31ca0a27931-tls-certs\") pod \"46dfc811-340c-4bc9-a07a-f31ca0a27931\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") "
Sep 08 14:56:43.428791 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.428735    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-tmp-dir\") pod \"46dfc811-340c-4bc9-a07a-f31ca0a27931\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") "
Sep 08 14:56:43.428791 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.428758    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68mc8\" (UniqueName: \"kubernetes.io/projected/46dfc811-340c-4bc9-a07a-f31ca0a27931-kube-api-access-68mc8\") pod \"46dfc811-340c-4bc9-a07a-f31ca0a27931\" (UID: \"46dfc811-340c-4bc9-a07a-f31ca0a27931\") "
Sep 08 14:56:43.429091 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.428976    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-model-cache" (OuterVolumeSpecName: "model-cache") pod "46dfc811-340c-4bc9-a07a-f31ca0a27931" (UID: "46dfc811-340c-4bc9-a07a-f31ca0a27931"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:56:43.429091 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.428984    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-home" (OuterVolumeSpecName: "home") pod "46dfc811-340c-4bc9-a07a-f31ca0a27931" (UID: "46dfc811-340c-4bc9-a07a-f31ca0a27931"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:56:43.429209 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.429082    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "46dfc811-340c-4bc9-a07a-f31ca0a27931" (UID: "46dfc811-340c-4bc9-a07a-f31ca0a27931"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:56:43.429209 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.429140    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:43.429209 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.429156    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:43.431054 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.431030    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-dshm" (OuterVolumeSpecName: "dshm") pod "46dfc811-340c-4bc9-a07a-f31ca0a27931" (UID: "46dfc811-340c-4bc9-a07a-f31ca0a27931"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:56:43.431167 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.431060    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46dfc811-340c-4bc9-a07a-f31ca0a27931-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "46dfc811-340c-4bc9-a07a-f31ca0a27931" (UID: "46dfc811-340c-4bc9-a07a-f31ca0a27931"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:56:43.431167 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.431073    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46dfc811-340c-4bc9-a07a-f31ca0a27931-kube-api-access-68mc8" (OuterVolumeSpecName: "kube-api-access-68mc8") pod "46dfc811-340c-4bc9-a07a-f31ca0a27931" (UID: "46dfc811-340c-4bc9-a07a-f31ca0a27931"). InnerVolumeSpecName "kube-api-access-68mc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:56:43.466653 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.466622    2583 generic.go:356] "Generic (PLEG): container finished" podID="d64fbb59-30c4-466e-bc27-a98234bffee4" containerID="6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d" exitCode=0
Sep 08 14:56:43.466813 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.466680    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"
Sep 08 14:56:43.466813 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.466720    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r" event={"ID":"d64fbb59-30c4-466e-bc27-a98234bffee4","Type":"ContainerDied","Data":"6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d"}
Sep 08 14:56:43.466813 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.466764    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r" event={"ID":"d64fbb59-30c4-466e-bc27-a98234bffee4","Type":"ContainerDied","Data":"2f164b6927103275e044c72bf119dba416a464ec148aa5fa443241ea7a17baec"}
Sep 08 14:56:43.466813 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.466783    2583 scope.go:117] "RemoveContainer" containerID="6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d"
Sep 08 14:56:43.468526 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.468500    2583 generic.go:356] "Generic (PLEG): container finished" podID="46dfc811-340c-4bc9-a07a-f31ca0a27931" containerID="3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a" exitCode=0
Sep 08 14:56:43.468622 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.468540    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r" event={"ID":"46dfc811-340c-4bc9-a07a-f31ca0a27931","Type":"ContainerDied","Data":"3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a"}
Sep 08 14:56:43.468622 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.468553    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"
Sep 08 14:56:43.468622 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.468577    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r" event={"ID":"46dfc811-340c-4bc9-a07a-f31ca0a27931","Type":"ContainerDied","Data":"d89f3b997c8e7da924fb7d4c169abd4ef991c4cf5fdd131c03721c9c5a1c2555"}
Sep 08 14:56:43.476523 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.476505    2583 scope.go:117] "RemoveContainer" containerID="6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d"
Sep 08 14:56:43.476875 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:56:43.476848    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d\": container with ID starting with 6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d not found: ID does not exist" containerID="6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d"
Sep 08 14:56:43.476992 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.476893    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d"} err="failed to get container status \"6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d\": rpc error: code = NotFound desc = could not find container \"6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d\": container with ID starting with 6536b27fa70df99346e701564a406303942fc91feaf4e7fe9f48b31245e33a6d not found: ID does not exist"
Sep 08 14:56:43.476992 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.476916    2583 scope.go:117] "RemoveContainer" containerID="3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a"
Sep 08 14:56:43.485132 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.485115    2583 scope.go:117] "RemoveContainer" containerID="3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a"
Sep 08 14:56:43.485410 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:56:43.485395    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a\": container with ID starting with 3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a not found: ID does not exist" containerID="3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a"
Sep 08 14:56:43.485450 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.485418    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a"} err="failed to get container status \"3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a\": rpc error: code = NotFound desc = could not find container \"3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a\": container with ID starting with 3df4acb6a01fc8d968adddce931ea22a0daa6c0fd34cbb8cca10a9bcc4c8332a not found: ID does not exist"
Sep 08 14:56:43.490063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.490039    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"]
Sep 08 14:56:43.494474 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.494452    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/sub-test-model-ef8f61-kserve-787f44787f-flr4r"]
Sep 08 14:56:43.505939 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.505911    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"]
Sep 08 14:56:43.509199 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.509175    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/sub-test-model-f83533-kserve-7597dc7d5f-7rj9r"]
Sep 08 14:56:43.529673 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.529643    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:43.529673 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.529667    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68mc8\" (UniqueName: \"kubernetes.io/projected/46dfc811-340c-4bc9-a07a-f31ca0a27931-kube-api-access-68mc8\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:43.529673 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.529676    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46dfc811-340c-4bc9-a07a-f31ca0a27931-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:43.529883 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.529684    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46dfc811-340c-4bc9-a07a-f31ca0a27931-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:43.794184 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.794153    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46dfc811-340c-4bc9-a07a-f31ca0a27931" path="/var/lib/kubelet/pods/46dfc811-340c-4bc9-a07a-f31ca0a27931/volumes"
Sep 08 14:56:43.794543 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:43.794530    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d64fbb59-30c4-466e-bc27-a98234bffee4" path="/var/lib/kubelet/pods/d64fbb59-30c4-466e-bc27-a98234bffee4/volumes"
Sep 08 14:56:44.179789 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:44.179704    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:44.179789 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:44.179753    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:44.192388 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:44.192366    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:44.487118 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:44.487091    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:56:47.279759 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:47.279707    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:47.812178 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:47.812143    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:47.838621 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:47.838587    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"]
Sep 08 14:56:47.838872 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:47.838823    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7" podUID="d4490b6d-08ad-40ce-ab85-d8c8f70645a5" containerName="payload-processing" containerID="cri-o://938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758" gracePeriod=30
Sep 08 14:56:47.848231 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:47.848207    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"]
Sep 08 14:56:47.848471 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:47.848445    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch" podUID="513df6e4-de9b-4481-a0e8-33be4a6a034c" containerName="payload-pre-processing" containerID="cri-o://5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896" gracePeriod=30
Sep 08 14:56:47.883266 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:47.883242    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:47.984900 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:47.984822    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:47.989815 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:47.989762    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"]
Sep 08 14:56:47.992649 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:47.992599    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1145ca-td62b"]
Sep 08 14:56:48.065378 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.065174    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:48.122445 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.122422    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:56:48.125297 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.125275    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:56:48.172958 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.172925    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-plugins-config-volume\") pod \"d4490b6d-08ad-40ce-ab85-d8c8f70645a5\" (UID: \"d4490b6d-08ad-40ce-ab85-d8c8f70645a5\") "
Sep 08 14:56:48.172958 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.172960    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513df6e4-de9b-4481-a0e8-33be4a6a034c-plugins-config-volume\") pod \"513df6e4-de9b-4481-a0e8-33be4a6a034c\" (UID: \"513df6e4-de9b-4481-a0e8-33be4a6a034c\") "
Sep 08 14:56:48.173208 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.173000    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8qbl\" (UniqueName: \"kubernetes.io/projected/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-kube-api-access-m8qbl\") pod \"d4490b6d-08ad-40ce-ab85-d8c8f70645a5\" (UID: \"d4490b6d-08ad-40ce-ab85-d8c8f70645a5\") "
Sep 08 14:56:48.173208 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.173041    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7ksk\" (UniqueName: \"kubernetes.io/projected/513df6e4-de9b-4481-a0e8-33be4a6a034c-kube-api-access-g7ksk\") pod \"513df6e4-de9b-4481-a0e8-33be4a6a034c\" (UID: \"513df6e4-de9b-4481-a0e8-33be4a6a034c\") "
Sep 08 14:56:48.173372 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.173347    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4490b6d-08ad-40ce-ab85-d8c8f70645a5" (UID: "d4490b6d-08ad-40ce-ab85-d8c8f70645a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:56:48.173503 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.173401    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/513df6e4-de9b-4481-a0e8-33be4a6a034c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "513df6e4-de9b-4481-a0e8-33be4a6a034c" (UID: "513df6e4-de9b-4481-a0e8-33be4a6a034c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:56:48.175299 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.175268    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-kube-api-access-m8qbl" (OuterVolumeSpecName: "kube-api-access-m8qbl") pod "d4490b6d-08ad-40ce-ab85-d8c8f70645a5" (UID: "d4490b6d-08ad-40ce-ab85-d8c8f70645a5"). InnerVolumeSpecName "kube-api-access-m8qbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:56:48.175388 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.175342    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/513df6e4-de9b-4481-a0e8-33be4a6a034c-kube-api-access-g7ksk" (OuterVolumeSpecName: "kube-api-access-g7ksk") pod "513df6e4-de9b-4481-a0e8-33be4a6a034c" (UID: "513df6e4-de9b-4481-a0e8-33be4a6a034c"). InnerVolumeSpecName "kube-api-access-g7ksk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:56:48.274048 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.274012    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:48.274048 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.274043    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513df6e4-de9b-4481-a0e8-33be4a6a034c-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:48.274048 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.274052    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8qbl\" (UniqueName: \"kubernetes.io/projected/d4490b6d-08ad-40ce-ab85-d8c8f70645a5-kube-api-access-m8qbl\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:48.274306 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.274061    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7ksk\" (UniqueName: \"kubernetes.io/projected/513df6e4-de9b-4481-a0e8-33be4a6a034c-kube-api-access-g7ksk\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:48.361095 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.361014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:48.507296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.507262    2583 generic.go:356] "Generic (PLEG): container finished" podID="d4490b6d-08ad-40ce-ab85-d8c8f70645a5" containerID="938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758" exitCode=0
Sep 08 14:56:48.507453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.507306    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"
Sep 08 14:56:48.507453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.507353    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7" event={"ID":"d4490b6d-08ad-40ce-ab85-d8c8f70645a5","Type":"ContainerDied","Data":"938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758"}
Sep 08 14:56:48.507453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.507398    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7" event={"ID":"d4490b6d-08ad-40ce-ab85-d8c8f70645a5","Type":"ContainerDied","Data":"0e41e64c7c2830dac5c5e9acd5252a297ff3a5a371b49c1b96013a37eb5178a6"}
Sep 08 14:56:48.507453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.507419    2583 scope.go:117] "RemoveContainer" containerID="938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758"
Sep 08 14:56:48.508747 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.508727    2583 generic.go:356] "Generic (PLEG): container finished" podID="513df6e4-de9b-4481-a0e8-33be4a6a034c" containerID="5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896" exitCode=0
Sep 08 14:56:48.509060 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.508756    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch" event={"ID":"513df6e4-de9b-4481-a0e8-33be4a6a034c","Type":"ContainerDied","Data":"5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896"}
Sep 08 14:56:48.509060 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.508771    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"
Sep 08 14:56:48.509060 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.508808    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch" event={"ID":"513df6e4-de9b-4481-a0e8-33be4a6a034c","Type":"ContainerDied","Data":"dfe21a095fee55b247670e5b86ee99e0c11eee4ce479d8948da590f609b3d70d"}
Sep 08 14:56:48.516808 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.516792    2583 scope.go:117] "RemoveContainer" containerID="938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758"
Sep 08 14:56:48.517142 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:56:48.517125    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758\": container with ID starting with 938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758 not found: ID does not exist" containerID="938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758"
Sep 08 14:56:48.517201 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.517151    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758"} err="failed to get container status \"938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758\": rpc error: code = NotFound desc = could not find container \"938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758\": container with ID starting with 938e5ec2e09419fce03c6a3d3e4d90657dfa2022f43e76f19a208225a7c75758 not found: ID does not exist"
Sep 08 14:56:48.517201 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.517167    2583 scope.go:117] "RemoveContainer" containerID="5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896"
Sep 08 14:56:48.525699 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.525682    2583 scope.go:117] "RemoveContainer" containerID="5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896"
Sep 08 14:56:48.525977 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:56:48.525958    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896\": container with ID starting with 5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896 not found: ID does not exist" containerID="5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896"
Sep 08 14:56:48.526045 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.525985    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896"} err="failed to get container status \"5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896\": rpc error: code = NotFound desc = could not find container \"5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896\": container with ID starting with 5bf893f5cc5b35e800e2cf998cde61602fa35ef283ac0077a23cb2d39c074896 not found: ID does not exist"
Sep 08 14:56:48.532768 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.532748    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"]
Sep 08 14:56:48.534781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.534763    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1145ca-56c794dcd6-w2kch"]
Sep 08 14:56:48.545081 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.545055    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"]
Sep 08 14:56:48.547379 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:48.547359    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1145ca-5d49df48c7-5wgj7"]
Sep 08 14:56:49.793183 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:49.793150    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0be3e1d0-1b56-43ad-9dda-a320ccd20197" path="/var/lib/kubelet/pods/0be3e1d0-1b56-43ad-9dda-a320ccd20197/volumes"
Sep 08 14:56:49.793528 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:49.793501    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="513df6e4-de9b-4481-a0e8-33be4a6a034c" path="/var/lib/kubelet/pods/513df6e4-de9b-4481-a0e8-33be4a6a034c/volumes"
Sep 08 14:56:49.793794 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:49.793781    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4490b6d-08ad-40ce-ab85-d8c8f70645a5" path="/var/lib/kubelet/pods/d4490b6d-08ad-40ce-ab85-d8c8f70645a5/volumes"
Sep 08 14:56:54.338365 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.338328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:54.412165 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.412100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:54.915561 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.915528    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"]
Sep 08 14:56:54.915964 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.915948    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be3e1d0-1b56-43ad-9dda-a320ccd20197" containerName="revoke-keys"
Sep 08 14:56:54.916014 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.915966    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be3e1d0-1b56-43ad-9dda-a320ccd20197" containerName="revoke-keys"
Sep 08 14:56:54.916014 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.915995    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46dfc811-340c-4bc9-a07a-f31ca0a27931" containerName="main"
Sep 08 14:56:54.916014 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916001    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="46dfc811-340c-4bc9-a07a-f31ca0a27931" containerName="main"
Sep 08 14:56:54.916014 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916008    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d64fbb59-30c4-466e-bc27-a98234bffee4" containerName="main"
Sep 08 14:56:54.916014 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916014    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d64fbb59-30c4-466e-bc27-a98234bffee4" containerName="main"
Sep 08 14:56:54.916172 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916020    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4490b6d-08ad-40ce-ab85-d8c8f70645a5" containerName="payload-processing"
Sep 08 14:56:54.916172 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916025    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4490b6d-08ad-40ce-ab85-d8c8f70645a5" containerName="payload-processing"
Sep 08 14:56:54.916172 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916037    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="513df6e4-de9b-4481-a0e8-33be4a6a034c" containerName="payload-pre-processing"
Sep 08 14:56:54.916172 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916041    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="513df6e4-de9b-4481-a0e8-33be4a6a034c" containerName="payload-pre-processing"
Sep 08 14:56:54.916172 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916107    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be3e1d0-1b56-43ad-9dda-a320ccd20197" containerName="revoke-keys"
Sep 08 14:56:54.916172 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916115    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4490b6d-08ad-40ce-ab85-d8c8f70645a5" containerName="payload-processing"
Sep 08 14:56:54.916172 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916124    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d64fbb59-30c4-466e-bc27-a98234bffee4" containerName="main"
Sep 08 14:56:54.916172 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916133    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="46dfc811-340c-4bc9-a07a-f31ca0a27931" containerName="main"
Sep 08 14:56:54.916172 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.916143    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="513df6e4-de9b-4481-a0e8-33be4a6a034c" containerName="payload-pre-processing"
Sep 08 14:56:54.920968 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.920951    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"
Sep 08 14:56:54.924248 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.924223    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:56:54.925091 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.925070    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 14:56:54.926911 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:54.926892    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"]
Sep 08 14:56:55.029819 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:55.029783    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9bc3ec0-40f1-4dd2-a084-cb7eaa827030-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz\" (UID: \"f9bc3ec0-40f1-4dd2-a084-cb7eaa827030\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"
Sep 08 14:56:55.130966 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:55.130929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9bc3ec0-40f1-4dd2-a084-cb7eaa827030-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz\" (UID: \"f9bc3ec0-40f1-4dd2-a084-cb7eaa827030\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"
Sep 08 14:56:55.131569 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:55.131551    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9bc3ec0-40f1-4dd2-a084-cb7eaa827030-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz\" (UID: \"f9bc3ec0-40f1-4dd2-a084-cb7eaa827030\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"
Sep 08 14:56:55.234260 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:55.234235    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"
Sep 08 14:56:55.509189 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:55.509111    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"]
Sep 08 14:56:55.512687 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:56:55.512652    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9bc3ec0_40f1_4dd2_a084_cb7eaa827030.slice/crio-18bf9b50ea39e0726fe2466cf427e4ea405af35b38e96017e592b5adaa2ab73d WatchSource:0}: Error finding container 18bf9b50ea39e0726fe2466cf427e4ea405af35b38e96017e592b5adaa2ab73d: Status 404 returned error can't find the container with id 18bf9b50ea39e0726fe2466cf427e4ea405af35b38e96017e592b5adaa2ab73d
Sep 08 14:56:55.549119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:55.549090    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz" event={"ID":"f9bc3ec0-40f1-4dd2-a084-cb7eaa827030","Type":"ContainerStarted","Data":"18bf9b50ea39e0726fe2466cf427e4ea405af35b38e96017e592b5adaa2ab73d"}
Sep 08 14:56:56.154093 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:56.154061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:56.561448 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:56.561409    2583 generic.go:356] "Generic (PLEG): container finished" podID="f9bc3ec0-40f1-4dd2-a084-cb7eaa827030" containerID="5c725d431cedda314d198e944d28d91f61eb20c3ec68967066ccfab1e9dbf217" exitCode=0
Sep 08 14:56:56.561875 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:56.561494    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz" event={"ID":"f9bc3ec0-40f1-4dd2-a084-cb7eaa827030","Type":"ContainerDied","Data":"5c725d431cedda314d198e944d28d91f61eb20c3ec68967066ccfab1e9dbf217"}
Sep 08 14:56:56.929125 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:56.929017    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:56:57.696734 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:57.696709    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"
Sep 08 14:56:57.756457 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:57.756429    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9bc3ec0-40f1-4dd2-a084-cb7eaa827030-maas-api-service-ca\") pod \"f9bc3ec0-40f1-4dd2-a084-cb7eaa827030\" (UID: \"f9bc3ec0-40f1-4dd2-a084-cb7eaa827030\") "
Sep 08 14:56:57.756796 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:57.756774    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9bc3ec0-40f1-4dd2-a084-cb7eaa827030-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f9bc3ec0-40f1-4dd2-a084-cb7eaa827030" (UID: "f9bc3ec0-40f1-4dd2-a084-cb7eaa827030"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:56:57.857641 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:57.857561    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9bc3ec0-40f1-4dd2-a084-cb7eaa827030-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:56:58.573193 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:58.573159    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"
Sep 08 14:56:58.573193 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:58.573187    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz" event={"ID":"f9bc3ec0-40f1-4dd2-a084-cb7eaa827030","Type":"ContainerDied","Data":"18bf9b50ea39e0726fe2466cf427e4ea405af35b38e96017e592b5adaa2ab73d"}
Sep 08 14:56:58.573409 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:56:58.573235    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18bf9b50ea39e0726fe2466cf427e4ea405af35b38e96017e592b5adaa2ab73d"
Sep 08 14:57:09.936215 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:09.936173    2583 scope.go:117] "RemoveContainer" containerID="1d22cc038f824a94f324b91d1c01a1bd314aaeb9d56af532e09dfe05c49d6e2f"
Sep 08 14:57:10.372532 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.372498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:10.400546 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.400518    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"]
Sep 08 14:57:10.400752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.400727    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8" podUID="50c83458-3ac9-47bb-ad69-7a5838f3e069" containerName="payload-pre-processing" containerID="cri-o://892ec8ea9933eb6461aa68c03008dc012f1460b44679a3769b8683658248b4f4" gracePeriod=30
Sep 08 14:57:10.472529 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.472499    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:10.537119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.537077    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"]
Sep 08 14:57:10.539148 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.539129    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4e2141-ht9bz"]
Sep 08 14:57:10.543997 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.543967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:10.634358 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.634325    2583 generic.go:356] "Generic (PLEG): container finished" podID="50c83458-3ac9-47bb-ad69-7a5838f3e069" containerID="892ec8ea9933eb6461aa68c03008dc012f1460b44679a3769b8683658248b4f4" exitCode=0
Sep 08 14:57:10.634535 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.634376    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8" event={"ID":"50c83458-3ac9-47bb-ad69-7a5838f3e069","Type":"ContainerDied","Data":"892ec8ea9933eb6461aa68c03008dc012f1460b44679a3769b8683658248b4f4"}
Sep 08 14:57:10.645953 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.645922    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:10.651912 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.651894    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:57:10.768649 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.768614    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50c83458-3ac9-47bb-ad69-7a5838f3e069-plugins-config-volume\") pod \"50c83458-3ac9-47bb-ad69-7a5838f3e069\" (UID: \"50c83458-3ac9-47bb-ad69-7a5838f3e069\") "
Sep 08 14:57:10.768814 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.768659    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k95ml\" (UniqueName: \"kubernetes.io/projected/50c83458-3ac9-47bb-ad69-7a5838f3e069-kube-api-access-k95ml\") pod \"50c83458-3ac9-47bb-ad69-7a5838f3e069\" (UID: \"50c83458-3ac9-47bb-ad69-7a5838f3e069\") "
Sep 08 14:57:10.769010 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.768990    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50c83458-3ac9-47bb-ad69-7a5838f3e069-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "50c83458-3ac9-47bb-ad69-7a5838f3e069" (UID: "50c83458-3ac9-47bb-ad69-7a5838f3e069"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:57:10.771083 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.771058    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50c83458-3ac9-47bb-ad69-7a5838f3e069-kube-api-access-k95ml" (OuterVolumeSpecName: "kube-api-access-k95ml") pod "50c83458-3ac9-47bb-ad69-7a5838f3e069" (UID: "50c83458-3ac9-47bb-ad69-7a5838f3e069"). InnerVolumeSpecName "kube-api-access-k95ml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:57:10.869916 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.869880    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50c83458-3ac9-47bb-ad69-7a5838f3e069-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:57:10.869916 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.869908    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k95ml\" (UniqueName: \"kubernetes.io/projected/50c83458-3ac9-47bb-ad69-7a5838f3e069-kube-api-access-k95ml\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:57:10.912374 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:10.912301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:11.641085 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:11.641050    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8" event={"ID":"50c83458-3ac9-47bb-ad69-7a5838f3e069","Type":"ContainerDied","Data":"3ffc53a373fb5b5b5396d51c3ec0c92b78fb38f65ead029a911993d92ed2df82"}
Sep 08 14:57:11.641085 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:11.641071    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"
Sep 08 14:57:11.641558 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:11.641118    2583 scope.go:117] "RemoveContainer" containerID="892ec8ea9933eb6461aa68c03008dc012f1460b44679a3769b8683658248b4f4"
Sep 08 14:57:11.662741 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:11.662716    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"]
Sep 08 14:57:11.668741 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:11.668718    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4e2141-6778cc6946-pb8x8"]
Sep 08 14:57:11.794576 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:11.794544    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50c83458-3ac9-47bb-ad69-7a5838f3e069" path="/var/lib/kubelet/pods/50c83458-3ac9-47bb-ad69-7a5838f3e069/volumes"
Sep 08 14:57:11.794923 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:11.794909    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9bc3ec0-40f1-4dd2-a084-cb7eaa827030" path="/var/lib/kubelet/pods/f9bc3ec0-40f1-4dd2-a084-cb7eaa827030/volumes"
Sep 08 14:57:13.685782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:13.685749    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:13.741157 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:13.741121    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:16.555725 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:16.555687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:17.173075 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.173036    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:17.273307 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.273264    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:17.485606 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.485574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:17.748246 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.748159    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"]
Sep 08 14:57:17.748730 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.748710    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50c83458-3ac9-47bb-ad69-7a5838f3e069" containerName="payload-pre-processing"
Sep 08 14:57:17.748804 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.748734    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="50c83458-3ac9-47bb-ad69-7a5838f3e069" containerName="payload-pre-processing"
Sep 08 14:57:17.748804 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.748762    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9bc3ec0-40f1-4dd2-a084-cb7eaa827030" containerName="revoke-keys"
Sep 08 14:57:17.748804 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.748771    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9bc3ec0-40f1-4dd2-a084-cb7eaa827030" containerName="revoke-keys"
Sep 08 14:57:17.749012 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.748929    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="50c83458-3ac9-47bb-ad69-7a5838f3e069" containerName="payload-pre-processing"
Sep 08 14:57:17.749012 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.748954    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9bc3ec0-40f1-4dd2-a084-cb7eaa827030" containerName="revoke-keys"
Sep 08 14:57:17.753604 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.753584    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:17.756508 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.756488    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"kube-root-ca.crt\""
Sep 08 14:57:17.756603 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.756517    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"openshift-service-ca.crt\""
Sep 08 14:57:17.757460 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.757444    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"default-dockercfg-mxzdm\""
Sep 08 14:57:17.757697 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.757670    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f83533\"/\"rate-test-model-f83533-kserve-self-signed-certs\""
Sep 08 14:57:17.762954 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.762934    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"]
Sep 08 14:57:17.783465 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.783444    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:17.848905 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.848873    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"]
Sep 08 14:57:17.852241 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.852222    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"
Sep 08 14:57:17.854781 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.854762    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 14:57:17.854880 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.854762    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:57:17.865626 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.865601    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"]
Sep 08 14:57:17.890198 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.890172    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:17.932817 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.932793    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zllwf\" (UniqueName: \"kubernetes.io/projected/b6df9c2a-6a8f-4387-870a-4101ecb07add-kube-api-access-zllwf\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:17.932953 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.932842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-tmp-dir\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:17.932953 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.932866    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-model-cache\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:17.932953 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.932926    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-dshm\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:17.933072 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.932959    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6df9c2a-6a8f-4387-870a-4101ecb07add-tls-certs\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:17.933072 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:17.932984    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-home\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.034394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.034312    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-model-cache\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.034394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.034360    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-dshm\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.034394 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.034396    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6df9c2a-6a8f-4387-870a-4101ecb07add-tls-certs\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.034674 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.034422    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-home\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.034674 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.034479    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c449194-8f69-4349-a1e8-4598900855a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw\" (UID: \"0c449194-8f69-4349-a1e8-4598900855a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"
Sep 08 14:57:18.034674 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.034541    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zllwf\" (UniqueName: \"kubernetes.io/projected/b6df9c2a-6a8f-4387-870a-4101ecb07add-kube-api-access-zllwf\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.034674 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.034570    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-tmp-dir\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.034932 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.034745    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-model-cache\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.034932 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.034761    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-home\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.035012 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.034929    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-tmp-dir\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.036742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.036721    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-dshm\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.037161 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.037142    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6df9c2a-6a8f-4387-870a-4101ecb07add-tls-certs\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.044495 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.044472    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zllwf\" (UniqueName: \"kubernetes.io/projected/b6df9c2a-6a8f-4387-870a-4101ecb07add-kube-api-access-zllwf\") pod \"rate-test-model-f83533-kserve-6744d6d6c-h7fcl\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") " pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.063391 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.063365    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:18.135972 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.135941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c449194-8f69-4349-a1e8-4598900855a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw\" (UID: \"0c449194-8f69-4349-a1e8-4598900855a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"
Sep 08 14:57:18.136811 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.136780    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c449194-8f69-4349-a1e8-4598900855a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw\" (UID: \"0c449194-8f69-4349-a1e8-4598900855a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"
Sep 08 14:57:18.162016 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.161991    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"
Sep 08 14:57:18.339554 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.339531    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"]
Sep 08 14:57:18.342033 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:57:18.341997    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6df9c2a_6a8f_4387_870a_4101ecb07add.slice/crio-3533ec66e9e70795a850d7138159662d146d1a4c522d9f9709706ccf2cc7267b WatchSource:0}: Error finding container 3533ec66e9e70795a850d7138159662d146d1a4c522d9f9709706ccf2cc7267b: Status 404 returned error can't find the container with id 3533ec66e9e70795a850d7138159662d146d1a4c522d9f9709706ccf2cc7267b
Sep 08 14:57:18.436810 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.436780    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"]
Sep 08 14:57:18.439409 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:57:18.439377    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c449194_8f69_4349_a1e8_4598900855a2.slice/crio-77503615e5eca7c9affac749eaa106b62f22eec7201c318b4ad0b9e39db9e434 WatchSource:0}: Error finding container 77503615e5eca7c9affac749eaa106b62f22eec7201c318b4ad0b9e39db9e434: Status 404 returned error can't find the container with id 77503615e5eca7c9affac749eaa106b62f22eec7201c318b4ad0b9e39db9e434
Sep 08 14:57:18.685256 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.685189    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw" event={"ID":"0c449194-8f69-4349-a1e8-4598900855a2","Type":"ContainerStarted","Data":"77503615e5eca7c9affac749eaa106b62f22eec7201c318b4ad0b9e39db9e434"}
Sep 08 14:57:18.686771 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.686744    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl" event={"ID":"b6df9c2a-6a8f-4387-870a-4101ecb07add","Type":"ContainerStarted","Data":"2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59"}
Sep 08 14:57:18.686907 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.686779    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl" event={"ID":"b6df9c2a-6a8f-4387-870a-4101ecb07add","Type":"ContainerStarted","Data":"3533ec66e9e70795a850d7138159662d146d1a4c522d9f9709706ccf2cc7267b"}
Sep 08 14:57:18.706431 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:18.706380    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl" podStartSLOduration=1.7063646860000001 podStartE2EDuration="1.706364686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:57:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:57:18.70344736 +0000 UTC m=+1389.431264371" watchObservedRunningTime="2026-09-08 14:57:18.706364686 +0000 UTC m=+1389.434181699"
Sep 08 14:57:19.693328 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:19.693292    2583 generic.go:356] "Generic (PLEG): container finished" podID="0c449194-8f69-4349-a1e8-4598900855a2" containerID="a4239a7e2d2bb5e2f8f1a7e4d2172df66385d5331f57547fbc52cbcf8ce1b398" exitCode=0
Sep 08 14:57:19.693721 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:19.693379    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw" event={"ID":"0c449194-8f69-4349-a1e8-4598900855a2","Type":"ContainerDied","Data":"a4239a7e2d2bb5e2f8f1a7e4d2172df66385d5331f57547fbc52cbcf8ce1b398"}
Sep 08 14:57:20.834266 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:20.834241    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"
Sep 08 14:57:20.962404 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:20.962327    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c449194-8f69-4349-a1e8-4598900855a2-maas-api-service-ca\") pod \"0c449194-8f69-4349-a1e8-4598900855a2\" (UID: \"0c449194-8f69-4349-a1e8-4598900855a2\") "
Sep 08 14:57:20.962665 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:20.962642    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c449194-8f69-4349-a1e8-4598900855a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c449194-8f69-4349-a1e8-4598900855a2" (UID: "0c449194-8f69-4349-a1e8-4598900855a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:57:21.063263 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:21.063227    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c449194-8f69-4349-a1e8-4598900855a2-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:57:21.705151 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:21.705123    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"
Sep 08 14:57:21.705322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:21.705162    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw" event={"ID":"0c449194-8f69-4349-a1e8-4598900855a2","Type":"ContainerDied","Data":"77503615e5eca7c9affac749eaa106b62f22eec7201c318b4ad0b9e39db9e434"}
Sep 08 14:57:21.705322 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:21.705213    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77503615e5eca7c9affac749eaa106b62f22eec7201c318b4ad0b9e39db9e434"
Sep 08 14:57:27.539215 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:27.539179    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:28.064259 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.064227    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:28.064453 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.064297    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:28.076990 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.076963    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:28.188931 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.188900    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:28.219795 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.219752    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"]
Sep 08 14:57:28.224419 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.224390    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"]
Sep 08 14:57:28.224684 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.224655    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p" podUID="f5fe1e20-f93f-4b50-9699-a61661df4b52" containerName="payload-pre-processing" containerID="cri-o://f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef" gracePeriod=30
Sep 08 14:57:28.289687 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.289651    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:28.405385 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.405347    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"]
Sep 08 14:57:28.407352 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.407309    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:28.420408 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.420385    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-11dfb4-tr2cw"]
Sep 08 14:57:28.479889 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.479862    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:57:28.526759 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.526735    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fe1e20-f93f-4b50-9699-a61661df4b52-plugins-config-volume\") pod \"f5fe1e20-f93f-4b50-9699-a61661df4b52\" (UID: \"f5fe1e20-f93f-4b50-9699-a61661df4b52\") "
Sep 08 14:57:28.526928 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.526810    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltlx2\" (UniqueName: \"kubernetes.io/projected/f5fe1e20-f93f-4b50-9699-a61661df4b52-kube-api-access-ltlx2\") pod \"f5fe1e20-f93f-4b50-9699-a61661df4b52\" (UID: \"f5fe1e20-f93f-4b50-9699-a61661df4b52\") "
Sep 08 14:57:28.527168 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.527143    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5fe1e20-f93f-4b50-9699-a61661df4b52-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f5fe1e20-f93f-4b50-9699-a61661df4b52" (UID: "f5fe1e20-f93f-4b50-9699-a61661df4b52"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:57:28.528957 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.528937    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5fe1e20-f93f-4b50-9699-a61661df4b52-kube-api-access-ltlx2" (OuterVolumeSpecName: "kube-api-access-ltlx2") pod "f5fe1e20-f93f-4b50-9699-a61661df4b52" (UID: "f5fe1e20-f93f-4b50-9699-a61661df4b52"). InnerVolumeSpecName "kube-api-access-ltlx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:57:28.628301 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.628204    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltlx2\" (UniqueName: \"kubernetes.io/projected/f5fe1e20-f93f-4b50-9699-a61661df4b52-kube-api-access-ltlx2\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:57:28.628301 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.628247    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fe1e20-f93f-4b50-9699-a61661df4b52-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:57:28.744901 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.744860    2583 generic.go:356] "Generic (PLEG): container finished" podID="f5fe1e20-f93f-4b50-9699-a61661df4b52" containerID="f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef" exitCode=0
Sep 08 14:57:28.745064 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.744913    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"
Sep 08 14:57:28.745064 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.744941    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p" event={"ID":"f5fe1e20-f93f-4b50-9699-a61661df4b52","Type":"ContainerDied","Data":"f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef"}
Sep 08 14:57:28.745064 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.744999    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p" event={"ID":"f5fe1e20-f93f-4b50-9699-a61661df4b52","Type":"ContainerDied","Data":"c08dccb57c93535a52c2f0a65904d6153447758ffe50e5296ff83473dd4012c7"}
Sep 08 14:57:28.745064 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.745024    2583 scope.go:117] "RemoveContainer" containerID="f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef"
Sep 08 14:57:28.745275 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.745097    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l" podUID="a0277548-02bb-4266-96cb-9fd882ffe3dc" containerName="payload-processing" containerID="cri-o://e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5" gracePeriod=30
Sep 08 14:57:28.764020 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.764000    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:57:28.780296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.780262    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l" podUID="a0277548-02bb-4266-96cb-9fd882ffe3dc" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.61:9004: connect: connection refused"
Sep 08 14:57:28.813473 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.813452    2583 scope.go:117] "RemoveContainer" containerID="f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef"
Sep 08 14:57:28.813868 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:57:28.813845    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef\": container with ID starting with f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef not found: ID does not exist" containerID="f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef"
Sep 08 14:57:28.813970 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.813875    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef"} err="failed to get container status \"f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef\": rpc error: code = NotFound desc = could not find container \"f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef\": container with ID starting with f46332dcac25772fb926284b76c4f84833f6d75589e980f1bb7ea9943dc18cef not found: ID does not exist"
Sep 08 14:57:28.828696 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.828671    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"]
Sep 08 14:57:28.832464 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.832441    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-11dfb4-78fcd44d4b-svp5p"]
Sep 08 14:57:28.988918 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:28.988895    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:57:29.032063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.032033    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0277548-02bb-4266-96cb-9fd882ffe3dc-plugins-config-volume\") pod \"a0277548-02bb-4266-96cb-9fd882ffe3dc\" (UID: \"a0277548-02bb-4266-96cb-9fd882ffe3dc\") "
Sep 08 14:57:29.032222 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.032100    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlb8z\" (UniqueName: \"kubernetes.io/projected/a0277548-02bb-4266-96cb-9fd882ffe3dc-kube-api-access-rlb8z\") pod \"a0277548-02bb-4266-96cb-9fd882ffe3dc\" (UID: \"a0277548-02bb-4266-96cb-9fd882ffe3dc\") "
Sep 08 14:57:29.032448 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.032429    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0277548-02bb-4266-96cb-9fd882ffe3dc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a0277548-02bb-4266-96cb-9fd882ffe3dc" (UID: "a0277548-02bb-4266-96cb-9fd882ffe3dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:57:29.034243 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.034220    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0277548-02bb-4266-96cb-9fd882ffe3dc-kube-api-access-rlb8z" (OuterVolumeSpecName: "kube-api-access-rlb8z") pod "a0277548-02bb-4266-96cb-9fd882ffe3dc" (UID: "a0277548-02bb-4266-96cb-9fd882ffe3dc"). InnerVolumeSpecName "kube-api-access-rlb8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:57:29.133274 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.133229    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0277548-02bb-4266-96cb-9fd882ffe3dc-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:57:29.133274 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.133267    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlb8z\" (UniqueName: \"kubernetes.io/projected/a0277548-02bb-4266-96cb-9fd882ffe3dc-kube-api-access-rlb8z\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:57:29.753525 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.753490    2583 generic.go:356] "Generic (PLEG): container finished" podID="a0277548-02bb-4266-96cb-9fd882ffe3dc" containerID="e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5" exitCode=0
Sep 08 14:57:29.753936 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.753530    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"
Sep 08 14:57:29.753936 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.753581    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l" event={"ID":"a0277548-02bb-4266-96cb-9fd882ffe3dc","Type":"ContainerDied","Data":"e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5"}
Sep 08 14:57:29.753936 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.753633    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l" event={"ID":"a0277548-02bb-4266-96cb-9fd882ffe3dc","Type":"ContainerDied","Data":"8654266df3d15074539f347cbc2ff82db508bd8d539cd12f478c0893e694ca9b"}
Sep 08 14:57:29.753936 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.753662    2583 scope.go:117] "RemoveContainer" containerID="e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5"
Sep 08 14:57:29.762618 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.762598    2583 scope.go:117] "RemoveContainer" containerID="e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5"
Sep 08 14:57:29.762895 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:57:29.762876    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5\": container with ID starting with e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5 not found: ID does not exist" containerID="e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5"
Sep 08 14:57:29.762970 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.762904    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5"} err="failed to get container status \"e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5\": rpc error: code = NotFound desc = could not find container \"e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5\": container with ID starting with e1c3e954536e59eccdf1cbf28a66be23eac600893224512cb1d4604cf88794e5 not found: ID does not exist"
Sep 08 14:57:29.776196 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.776173    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"]
Sep 08 14:57:29.780006 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.779985    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-11dfb4-74b7dcb565-9b79l"]
Sep 08 14:57:29.793534 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.793510    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c449194-8f69-4349-a1e8-4598900855a2" path="/var/lib/kubelet/pods/0c449194-8f69-4349-a1e8-4598900855a2/volumes"
Sep 08 14:57:29.793880 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.793866    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0277548-02bb-4266-96cb-9fd882ffe3dc" path="/var/lib/kubelet/pods/a0277548-02bb-4266-96cb-9fd882ffe3dc/volumes"
Sep 08 14:57:29.794183 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:29.794171    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5fe1e20-f93f-4b50-9699-a61661df4b52" path="/var/lib/kubelet/pods/f5fe1e20-f93f-4b50-9699-a61661df4b52/volumes"
Sep 08 14:57:30.364165 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:30.364128    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:30.426967 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:30.426930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:35.346600 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.346567    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"]
Sep 08 14:57:35.347019 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.346963    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c449194-8f69-4349-a1e8-4598900855a2" containerName="revoke-keys"
Sep 08 14:57:35.347019 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.346976    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c449194-8f69-4349-a1e8-4598900855a2" containerName="revoke-keys"
Sep 08 14:57:35.347019 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.346988    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5fe1e20-f93f-4b50-9699-a61661df4b52" containerName="payload-pre-processing"
Sep 08 14:57:35.347019 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.346994    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5fe1e20-f93f-4b50-9699-a61661df4b52" containerName="payload-pre-processing"
Sep 08 14:57:35.347019 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.347014    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0277548-02bb-4266-96cb-9fd882ffe3dc" containerName="payload-processing"
Sep 08 14:57:35.347019 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.347019    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0277548-02bb-4266-96cb-9fd882ffe3dc" containerName="payload-processing"
Sep 08 14:57:35.347257 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.347077    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0277548-02bb-4266-96cb-9fd882ffe3dc" containerName="payload-processing"
Sep 08 14:57:35.347257 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.347087    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5fe1e20-f93f-4b50-9699-a61661df4b52" containerName="payload-pre-processing"
Sep 08 14:57:35.347257 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.347092    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c449194-8f69-4349-a1e8-4598900855a2" containerName="revoke-keys"
Sep 08 14:57:35.351633 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.351616    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"
Sep 08 14:57:35.354583 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.354559    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:57:35.354583 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.354574    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 14:57:35.358442 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.358404    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"]
Sep 08 14:57:35.392002 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.391978    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95eb8695-39d2-405b-a36b-5e0459e14b4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs\" (UID: \"95eb8695-39d2-405b-a36b-5e0459e14b4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"
Sep 08 14:57:35.493419 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.493392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95eb8695-39d2-405b-a36b-5e0459e14b4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs\" (UID: \"95eb8695-39d2-405b-a36b-5e0459e14b4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"
Sep 08 14:57:35.494061 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.494043    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95eb8695-39d2-405b-a36b-5e0459e14b4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs\" (UID: \"95eb8695-39d2-405b-a36b-5e0459e14b4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"
Sep 08 14:57:35.663309 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.663237    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"
Sep 08 14:57:35.935950 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:35.935861    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"]
Sep 08 14:57:35.939095 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:57:35.939060    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95eb8695_39d2_405b_a36b_5e0459e14b4d.slice/crio-966e0f69279c0a729a17969bf0fb04ffac5e823621a950673b4beb47586e690e WatchSource:0}: Error finding container 966e0f69279c0a729a17969bf0fb04ffac5e823621a950673b4beb47586e690e: Status 404 returned error can't find the container with id 966e0f69279c0a729a17969bf0fb04ffac5e823621a950673b4beb47586e690e
Sep 08 14:57:36.445855 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:36.445804    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:36.504346 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:36.504320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:36.794092 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:36.794058    2583 generic.go:356] "Generic (PLEG): container finished" podID="95eb8695-39d2-405b-a36b-5e0459e14b4d" containerID="328ef3ea92ae12b12e690bbedb08b9f5fdac8d6571cb7186bcedec9533bd07a0" exitCode=0
Sep 08 14:57:36.794247 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:36.794147    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs" event={"ID":"95eb8695-39d2-405b-a36b-5e0459e14b4d","Type":"ContainerDied","Data":"328ef3ea92ae12b12e690bbedb08b9f5fdac8d6571cb7186bcedec9533bd07a0"}
Sep 08 14:57:36.794247 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:36.794212    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs" event={"ID":"95eb8695-39d2-405b-a36b-5e0459e14b4d","Type":"ContainerStarted","Data":"966e0f69279c0a729a17969bf0fb04ffac5e823621a950673b4beb47586e690e"}
Sep 08 14:57:37.934011 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:37.933985    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"
Sep 08 14:57:38.017713 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:38.017686    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95eb8695-39d2-405b-a36b-5e0459e14b4d-maas-api-service-ca\") pod \"95eb8695-39d2-405b-a36b-5e0459e14b4d\" (UID: \"95eb8695-39d2-405b-a36b-5e0459e14b4d\") "
Sep 08 14:57:38.018082 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:38.018058    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95eb8695-39d2-405b-a36b-5e0459e14b4d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "95eb8695-39d2-405b-a36b-5e0459e14b4d" (UID: "95eb8695-39d2-405b-a36b-5e0459e14b4d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:57:38.119073 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:38.119004    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95eb8695-39d2-405b-a36b-5e0459e14b4d-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:57:38.805683 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:38.805656    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"
Sep 08 14:57:38.805873 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:38.805691    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs" event={"ID":"95eb8695-39d2-405b-a36b-5e0459e14b4d","Type":"ContainerDied","Data":"966e0f69279c0a729a17969bf0fb04ffac5e823621a950673b4beb47586e690e"}
Sep 08 14:57:38.805873 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:38.805739    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="966e0f69279c0a729a17969bf0fb04ffac5e823621a950673b4beb47586e690e"
Sep 08 14:57:45.870203 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:45.870159    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:45.970742 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:45.970714    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:46.036014 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:46.035981    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"]
Sep 08 14:57:46.038930 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:46.038904    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f8c7bd-djvzs"]
Sep 08 14:57:46.046080 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:46.046056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:46.117489 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:46.117461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:46.413276 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:46.413242    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:47.793124 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:47.793091    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95eb8695-39d2-405b-a36b-5e0459e14b4d" path="/var/lib/kubelet/pods/95eb8695-39d2-405b-a36b-5e0459e14b4d/volumes"
Sep 08 14:57:52.070431 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:52.070399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:52.141176 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:52.141142    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:52.331367 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:52.331293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:57:58.850195 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:57:58.850150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:01.974156 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:01.974071    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"]
Sep 08 14:58:02.004518 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:02.004488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:03.076978 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:03.076948    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"]
Sep 08 14:58:03.251510 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:03.251475    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:03.545011 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:03.544976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:05.566025 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:05.565991    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:05.667978 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:05.667947    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:06.712717 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:06.712687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:06.901444 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:06.901417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:08.011247 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:08.011199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:08.076641 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:08.076609    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:08.369617 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:08.369528    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:08.604817 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:08.604784    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:08.692498 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:08.692405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:16.996311 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:16.996249    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq" podUID="0aa948a8-d77a-4115-811e-25f2743607ef" containerName="main" containerID="cri-o://6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c" gracePeriod=45
Sep 08 14:58:17.250853 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.250781    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:58:17.369330 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.369290    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsvtm\" (UniqueName: \"kubernetes.io/projected/0aa948a8-d77a-4115-811e-25f2743607ef-kube-api-access-gsvtm\") pod \"0aa948a8-d77a-4115-811e-25f2743607ef\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") "
Sep 08 14:58:17.369531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.369339    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-tmp-dir\") pod \"0aa948a8-d77a-4115-811e-25f2743607ef\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") "
Sep 08 14:58:17.369531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.369416    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-model-cache\") pod \"0aa948a8-d77a-4115-811e-25f2743607ef\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") "
Sep 08 14:58:17.369531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.369438    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa948a8-d77a-4115-811e-25f2743607ef-tls-certs\") pod \"0aa948a8-d77a-4115-811e-25f2743607ef\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") "
Sep 08 14:58:17.369531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.369464    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-dshm\") pod \"0aa948a8-d77a-4115-811e-25f2743607ef\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") "
Sep 08 14:58:17.369531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.369521    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-home\") pod \"0aa948a8-d77a-4115-811e-25f2743607ef\" (UID: \"0aa948a8-d77a-4115-811e-25f2743607ef\") "
Sep 08 14:58:17.369782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.369673    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-model-cache" (OuterVolumeSpecName: "model-cache") pod "0aa948a8-d77a-4115-811e-25f2743607ef" (UID: "0aa948a8-d77a-4115-811e-25f2743607ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:58:17.369782 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.369680    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0aa948a8-d77a-4115-811e-25f2743607ef" (UID: "0aa948a8-d77a-4115-811e-25f2743607ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:58:17.369929 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.369872    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-home" (OuterVolumeSpecName: "home") pod "0aa948a8-d77a-4115-811e-25f2743607ef" (UID: "0aa948a8-d77a-4115-811e-25f2743607ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:58:17.369929 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.369881    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:17.369929 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.369926    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:17.371676 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.371651    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa948a8-d77a-4115-811e-25f2743607ef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0aa948a8-d77a-4115-811e-25f2743607ef" (UID: "0aa948a8-d77a-4115-811e-25f2743607ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:58:17.372119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.372098    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa948a8-d77a-4115-811e-25f2743607ef-kube-api-access-gsvtm" (OuterVolumeSpecName: "kube-api-access-gsvtm") pod "0aa948a8-d77a-4115-811e-25f2743607ef" (UID: "0aa948a8-d77a-4115-811e-25f2743607ef"). InnerVolumeSpecName "kube-api-access-gsvtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:58:17.372119 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.372108    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-dshm" (OuterVolumeSpecName: "dshm") pod "0aa948a8-d77a-4115-811e-25f2743607ef" (UID: "0aa948a8-d77a-4115-811e-25f2743607ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:58:17.470662 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.470628    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsvtm\" (UniqueName: \"kubernetes.io/projected/0aa948a8-d77a-4115-811e-25f2743607ef-kube-api-access-gsvtm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:17.470662 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.470660    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa948a8-d77a-4115-811e-25f2743607ef-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:17.470662 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.470670    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:17.470911 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:17.470677    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa948a8-d77a-4115-811e-25f2743607ef-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:18.018125 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.018092    2583 generic.go:356] "Generic (PLEG): container finished" podID="0aa948a8-d77a-4115-811e-25f2743607ef" containerID="6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c" exitCode=0
Sep 08 14:58:18.018572 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.018137    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"
Sep 08 14:58:18.018572 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.018182    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq" event={"ID":"0aa948a8-d77a-4115-811e-25f2743607ef","Type":"ContainerDied","Data":"6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c"}
Sep 08 14:58:18.018572 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.018223    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq" event={"ID":"0aa948a8-d77a-4115-811e-25f2743607ef","Type":"ContainerDied","Data":"c6ce7d3a4a40552cf70c2c7adf04b9eac8a6edf2340c5f35383feea7e5a9dd48"}
Sep 08 14:58:18.018572 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.018240    2583 scope.go:117] "RemoveContainer" containerID="6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c"
Sep 08 14:58:18.026864 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.026848    2583 scope.go:117] "RemoveContainer" containerID="6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c"
Sep 08 14:58:18.027163 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:58:18.027146    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c\": container with ID starting with 6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c not found: ID does not exist" containerID="6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c"
Sep 08 14:58:18.027220 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.027172    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c"} err="failed to get container status \"6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c\": rpc error: code = NotFound desc = could not find container \"6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c\": container with ID starting with 6a4192890b13993ffebb06ac197003cfd14d3373c0cc9f0f2f4534728bfc027c not found: ID does not exist"
Sep 08 14:58:18.035292 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.035269    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"]
Sep 08 14:58:18.039277 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.039256    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ef8f61/rate-test-model-ef8f61-kserve-58c69d99d7-rnptq"]
Sep 08 14:58:18.099405 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.099324    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl" podUID="b6df9c2a-6a8f-4387-870a-4101ecb07add" containerName="main" containerID="cri-o://2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59" gracePeriod=45
Sep 08 14:58:18.339791 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.339765    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:58:18.378131 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.378058    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-home\") pod \"b6df9c2a-6a8f-4387-870a-4101ecb07add\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") "
Sep 08 14:58:18.378131 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.378104    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zllwf\" (UniqueName: \"kubernetes.io/projected/b6df9c2a-6a8f-4387-870a-4101ecb07add-kube-api-access-zllwf\") pod \"b6df9c2a-6a8f-4387-870a-4101ecb07add\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") "
Sep 08 14:58:18.378131 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.378125    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-model-cache\") pod \"b6df9c2a-6a8f-4387-870a-4101ecb07add\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") "
Sep 08 14:58:18.378391 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.378160    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-dshm\") pod \"b6df9c2a-6a8f-4387-870a-4101ecb07add\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") "
Sep 08 14:58:18.378391 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.378185    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6df9c2a-6a8f-4387-870a-4101ecb07add-tls-certs\") pod \"b6df9c2a-6a8f-4387-870a-4101ecb07add\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") "
Sep 08 14:58:18.378391 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.378211    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-tmp-dir\") pod \"b6df9c2a-6a8f-4387-870a-4101ecb07add\" (UID: \"b6df9c2a-6a8f-4387-870a-4101ecb07add\") "
Sep 08 14:58:18.378391 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.378337    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-home" (OuterVolumeSpecName: "home") pod "b6df9c2a-6a8f-4387-870a-4101ecb07add" (UID: "b6df9c2a-6a8f-4387-870a-4101ecb07add"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:58:18.378601 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.378508    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-model-cache" (OuterVolumeSpecName: "model-cache") pod "b6df9c2a-6a8f-4387-870a-4101ecb07add" (UID: "b6df9c2a-6a8f-4387-870a-4101ecb07add"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:58:18.378601 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.378568    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:18.378601 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.378578    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b6df9c2a-6a8f-4387-870a-4101ecb07add" (UID: "b6df9c2a-6a8f-4387-870a-4101ecb07add"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:58:18.380458 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.380436    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-dshm" (OuterVolumeSpecName: "dshm") pod "b6df9c2a-6a8f-4387-870a-4101ecb07add" (UID: "b6df9c2a-6a8f-4387-870a-4101ecb07add"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:58:18.380458 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.380439    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6df9c2a-6a8f-4387-870a-4101ecb07add-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b6df9c2a-6a8f-4387-870a-4101ecb07add" (UID: "b6df9c2a-6a8f-4387-870a-4101ecb07add"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:58:18.380776 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.380751    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6df9c2a-6a8f-4387-870a-4101ecb07add-kube-api-access-zllwf" (OuterVolumeSpecName: "kube-api-access-zllwf") pod "b6df9c2a-6a8f-4387-870a-4101ecb07add" (UID: "b6df9c2a-6a8f-4387-870a-4101ecb07add"). InnerVolumeSpecName "kube-api-access-zllwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:58:18.479384 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.479350    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zllwf\" (UniqueName: \"kubernetes.io/projected/b6df9c2a-6a8f-4387-870a-4101ecb07add-kube-api-access-zllwf\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:18.479384 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.479379    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:18.479384 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.479389    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:18.479596 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.479397    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6df9c2a-6a8f-4387-870a-4101ecb07add-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:18.479596 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:18.479407    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6df9c2a-6a8f-4387-870a-4101ecb07add-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 14:58:19.024783 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:19.024748    2583 generic.go:356] "Generic (PLEG): container finished" podID="b6df9c2a-6a8f-4387-870a-4101ecb07add" containerID="2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59" exitCode=0
Sep 08 14:58:19.025260 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:19.024791    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"
Sep 08 14:58:19.025260 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:19.024858    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl" event={"ID":"b6df9c2a-6a8f-4387-870a-4101ecb07add","Type":"ContainerDied","Data":"2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59"}
Sep 08 14:58:19.025260 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:19.024914    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl" event={"ID":"b6df9c2a-6a8f-4387-870a-4101ecb07add","Type":"ContainerDied","Data":"3533ec66e9e70795a850d7138159662d146d1a4c522d9f9709706ccf2cc7267b"}
Sep 08 14:58:19.025260 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:19.024941    2583 scope.go:117] "RemoveContainer" containerID="2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59"
Sep 08 14:58:19.034764 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:19.034744    2583 scope.go:117] "RemoveContainer" containerID="2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59"
Sep 08 14:58:19.035064 ip-10-0-137-6 kubenswrapper[2583]: E0908 14:58:19.035046    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59\": container with ID starting with 2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59 not found: ID does not exist" containerID="2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59"
Sep 08 14:58:19.035128 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:19.035070    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59"} err="failed to get container status \"2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59\": rpc error: code = NotFound desc = could not find container \"2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59\": container with ID starting with 2d70463badb36fdf881ff76f5e8a5f48b68e4b4e28edb3fac16fef291b5f4e59 not found: ID does not exist"
Sep 08 14:58:19.048318 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:19.048295    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"]
Sep 08 14:58:19.058431 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:19.054522    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f83533/rate-test-model-f83533-kserve-6744d6d6c-h7fcl"]
Sep 08 14:58:19.793683 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:19.793646    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aa948a8-d77a-4115-811e-25f2743607ef" path="/var/lib/kubelet/pods/0aa948a8-d77a-4115-811e-25f2743607ef/volumes"
Sep 08 14:58:19.794056 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:19.794042    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6df9c2a-6a8f-4387-870a-4101ecb07add" path="/var/lib/kubelet/pods/b6df9c2a-6a8f-4387-870a-4101ecb07add/volumes"
Sep 08 14:58:31.772636 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:31.772604    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:32.514826 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:32.514791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:32.934709 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:32.934625    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:33.044101 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:33.044066    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:35.772253 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:35.772215    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:36.004235 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:36.004196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:36.917031 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:36.916992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:37.083352 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.083304    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:37.363515 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.363486    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:37.460567 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.460535    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:37.495972 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.495940    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"]
Sep 08 14:58:37.496318 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.496306    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95eb8695-39d2-405b-a36b-5e0459e14b4d" containerName="revoke-keys"
Sep 08 14:58:37.496362 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.496319    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="95eb8695-39d2-405b-a36b-5e0459e14b4d" containerName="revoke-keys"
Sep 08 14:58:37.496362 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.496329    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa948a8-d77a-4115-811e-25f2743607ef" containerName="main"
Sep 08 14:58:37.496362 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.496335    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa948a8-d77a-4115-811e-25f2743607ef" containerName="main"
Sep 08 14:58:37.496362 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.496342    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6df9c2a-6a8f-4387-870a-4101ecb07add" containerName="main"
Sep 08 14:58:37.496362 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.496348    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6df9c2a-6a8f-4387-870a-4101ecb07add" containerName="main"
Sep 08 14:58:37.496531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.496410    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="95eb8695-39d2-405b-a36b-5e0459e14b4d" containerName="revoke-keys"
Sep 08 14:58:37.496531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.496420    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6df9c2a-6a8f-4387-870a-4101ecb07add" containerName="main"
Sep 08 14:58:37.496531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.496426    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa948a8-d77a-4115-811e-25f2743607ef" containerName="main"
Sep 08 14:58:37.500873 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.500857    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 14:58:37.503719 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.503698    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-a89cbb\""
Sep 08 14:58:37.503875 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.503699    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-a89cbb-dockercfg-g64mc\""
Sep 08 14:58:37.508531 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.508169    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"]
Sep 08 14:58:37.549659 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.549632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:37.550477 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.550458    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"]
Sep 08 14:58:37.553752 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.553738    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 14:58:37.563910 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.563890    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"]
Sep 08 14:58:37.654166 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.654093    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7msdn\" (UniqueName: \"kubernetes.io/projected/434c7944-0250-4496-8249-559af445c24e-kube-api-access-7msdn\") pod \"payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4\" (UID: \"434c7944-0250-4496-8249-559af445c24e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 14:58:37.654166 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.654132    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434c7944-0250-4496-8249-559af445c24e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4\" (UID: \"434c7944-0250-4496-8249-559af445c24e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 14:58:37.654340 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.654165    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce7a1447-da27-4e1a-af1d-847b6128c28f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t\" (UID: \"ce7a1447-da27-4e1a-af1d-847b6128c28f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 14:58:37.654340 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.654245    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4bjc\" (UniqueName: \"kubernetes.io/projected/ce7a1447-da27-4e1a-af1d-847b6128c28f-kube-api-access-b4bjc\") pod \"payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t\" (UID: \"ce7a1447-da27-4e1a-af1d-847b6128c28f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 14:58:37.754926 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.754892    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7msdn\" (UniqueName: \"kubernetes.io/projected/434c7944-0250-4496-8249-559af445c24e-kube-api-access-7msdn\") pod \"payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4\" (UID: \"434c7944-0250-4496-8249-559af445c24e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 14:58:37.755063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.754936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434c7944-0250-4496-8249-559af445c24e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4\" (UID: \"434c7944-0250-4496-8249-559af445c24e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 14:58:37.755063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.754964    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce7a1447-da27-4e1a-af1d-847b6128c28f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t\" (UID: \"ce7a1447-da27-4e1a-af1d-847b6128c28f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 14:58:37.755063 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.755015    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4bjc\" (UniqueName: \"kubernetes.io/projected/ce7a1447-da27-4e1a-af1d-847b6128c28f-kube-api-access-b4bjc\") pod \"payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t\" (UID: \"ce7a1447-da27-4e1a-af1d-847b6128c28f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 14:58:37.755571 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.755553    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434c7944-0250-4496-8249-559af445c24e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4\" (UID: \"434c7944-0250-4496-8249-559af445c24e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 14:58:37.755681 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.755662    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce7a1447-da27-4e1a-af1d-847b6128c28f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t\" (UID: \"ce7a1447-da27-4e1a-af1d-847b6128c28f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 14:58:37.764194 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.764163    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7msdn\" (UniqueName: \"kubernetes.io/projected/434c7944-0250-4496-8249-559af445c24e-kube-api-access-7msdn\") pod \"payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4\" (UID: \"434c7944-0250-4496-8249-559af445c24e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 14:58:37.764296 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.764246    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4bjc\" (UniqueName: \"kubernetes.io/projected/ce7a1447-da27-4e1a-af1d-847b6128c28f-kube-api-access-b4bjc\") pod \"payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t\" (UID: \"ce7a1447-da27-4e1a-af1d-847b6128c28f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 14:58:37.812730 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.812704    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 14:58:37.864083 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:37.864018    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 14:58:38.092079 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:38.089362    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"]
Sep 08 14:58:38.121450 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:38.121420    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4" event={"ID":"434c7944-0250-4496-8249-559af445c24e","Type":"ContainerStarted","Data":"d6618689c34a56808900eb398dc1a7a990608a5ac92d6facb1e6f7a051cf276b"}
Sep 08 14:58:38.145177 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:38.145151    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"]
Sep 08 14:58:38.146546 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:58:38.146522    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce7a1447_da27_4e1a_af1d_847b6128c28f.slice/crio-f3b13b5a2c50fdcceff57213bee0ad1e03747b71369fe4f2328a26dfd35b9ade WatchSource:0}: Error finding container f3b13b5a2c50fdcceff57213bee0ad1e03747b71369fe4f2328a26dfd35b9ade: Status 404 returned error can't find the container with id f3b13b5a2c50fdcceff57213bee0ad1e03747b71369fe4f2328a26dfd35b9ade
Sep 08 14:58:39.128081 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:39.128043    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4" event={"ID":"434c7944-0250-4496-8249-559af445c24e","Type":"ContainerStarted","Data":"2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276"}
Sep 08 14:58:39.128532 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:39.128108    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 14:58:39.129575 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:39.129547    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t" event={"ID":"ce7a1447-da27-4e1a-af1d-847b6128c28f","Type":"ContainerStarted","Data":"0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09"}
Sep 08 14:58:39.129684 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:39.129600    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t" event={"ID":"ce7a1447-da27-4e1a-af1d-847b6128c28f","Type":"ContainerStarted","Data":"f3b13b5a2c50fdcceff57213bee0ad1e03747b71369fe4f2328a26dfd35b9ade"}
Sep 08 14:58:39.129684 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:39.129665    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 14:58:39.150478 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:39.150436    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4" podStartSLOduration=2.150423028 podStartE2EDuration="2.150423028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:58:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:58:39.147713558 +0000 UTC m=+1469.875530574" watchObservedRunningTime="2026-09-08 14:58:39.150423028 +0000 UTC m=+1469.878240041"
Sep 08 14:58:39.167995 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:39.167954    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t" podStartSLOduration=2.1679372949999998 podStartE2EDuration="2.167937295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:58:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:58:39.166112178 +0000 UTC m=+1469.893929193" watchObservedRunningTime="2026-09-08 14:58:39.167937295 +0000 UTC m=+1469.895754312"
Sep 08 14:58:49.871521 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:49.871484    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:50.135963 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:50.135882    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 14:58:50.135963 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:50.135951    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 14:58:51.611375 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:51.611340    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:58:57.789187 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:58:57.789154    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:59:00.836960 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:00.836924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:59:06.119018 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:06.118982    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t_ce7a1447-da27-4e1a-af1d-847b6128c28f/payload-processing/0.log"
Sep 08 14:59:09.993505 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:09.993472    2583 scope.go:117] "RemoveContainer" containerID="820c1b28fac357e0122374ddf9edad493e262b259342df817ab15230651fe19c"
Sep 08 14:59:23.064971 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.064935    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"]
Sep 08 14:59:23.070185 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.070160    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.074177 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.074153    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-28eb2c\"/\"ipp-route-28eb2c-kserve-self-signed-certs\""
Sep 08 14:59:23.074294 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.074159    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-28eb2c\"/\"kube-root-ca.crt\""
Sep 08 14:59:23.074294 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.074273    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-28eb2c\"/\"openshift-service-ca.crt\""
Sep 08 14:59:23.074423 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.074273    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-28eb2c\"/\"default-dockercfg-stl72\""
Sep 08 14:59:23.079094 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.079071    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"]
Sep 08 14:59:23.161075 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.161039    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-tmp-dir\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.161249 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.161101    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmcn7\" (UniqueName: \"kubernetes.io/projected/1781cc54-80a5-48c0-9bfb-efd5515ef928-kube-api-access-cmcn7\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.161249 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.161156    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-dshm\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.161249 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.161187    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-home\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.161249 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.161234    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-model-cache\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.161410 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.161266    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1781cc54-80a5-48c0-9bfb-efd5515ef928-tls-certs\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.261906 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.261870    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1781cc54-80a5-48c0-9bfb-efd5515ef928-tls-certs\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.262097 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.261932    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-tmp-dir\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.262097 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.261973    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmcn7\" (UniqueName: \"kubernetes.io/projected/1781cc54-80a5-48c0-9bfb-efd5515ef928-kube-api-access-cmcn7\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.262097 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.261991    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-dshm\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.262097 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.262009    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-home\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.262097 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.262044    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-model-cache\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.262436 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.262414    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-home\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.262497 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.262464    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-tmp-dir\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.262497 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.262489    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-model-cache\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.264437 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.264420    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-dshm\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.264584 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.264567    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1781cc54-80a5-48c0-9bfb-efd5515ef928-tls-certs\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.271126 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.271103    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmcn7\" (UniqueName: \"kubernetes.io/projected/1781cc54-80a5-48c0-9bfb-efd5515ef928-kube-api-access-cmcn7\") pod \"ipp-route-28eb2c-kserve-5fccc44467-vq9wf\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") " pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.381192 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.381120    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:23.651201 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:23.651177    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"]
Sep 08 14:59:23.653313 ip-10-0-137-6 kubenswrapper[2583]: W0908 14:59:23.653285    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1781cc54_80a5_48c0_9bfb_efd5515ef928.slice/crio-d1879935b04f2a8def26dfd378f458dfa4ad14e877847fc45c83ab59729c8c69 WatchSource:0}: Error finding container d1879935b04f2a8def26dfd378f458dfa4ad14e877847fc45c83ab59729c8c69: Status 404 returned error can't find the container with id d1879935b04f2a8def26dfd378f458dfa4ad14e877847fc45c83ab59729c8c69
Sep 08 14:59:24.001486 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:24.001454    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:59:24.368567 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:24.368482    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf" event={"ID":"1781cc54-80a5-48c0-9bfb-efd5515ef928","Type":"ContainerStarted","Data":"f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c"}
Sep 08 14:59:24.368567 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:24.368522    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf" event={"ID":"1781cc54-80a5-48c0-9bfb-efd5515ef928","Type":"ContainerStarted","Data":"d1879935b04f2a8def26dfd378f458dfa4ad14e877847fc45c83ab59729c8c69"}
Sep 08 14:59:24.387099 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:24.387058    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf" podStartSLOduration=1.387044514 podStartE2EDuration="1.387044514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:59:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:59:24.385170707 +0000 UTC m=+1515.112987721" watchObservedRunningTime="2026-09-08 14:59:24.387044514 +0000 UTC m=+1515.114861527"
Sep 08 14:59:33.381285 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:33.381255    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:33.383699 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:33.381311    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:33.394072 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:33.394044    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:33.426356 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:33.426324    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 14:59:41.174299 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:41.174261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:59:54.573961 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:54.573934    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t_ce7a1447-da27-4e1a-af1d-847b6128c28f/payload-processing/0.log"
Sep 08 14:59:54.873990 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:54.873914    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:59:56.881192 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:56.881165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:59:58.281347 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:58.281310    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:59:58.376130 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:58.376099    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:59:59.514797 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:59.514764    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:59:59.611027 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:59.610981    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 14:59:59.864789 ip-10-0-137-6 kubenswrapper[2583]: I0908 14:59:59.864708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:00.103714 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.103681    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"]
Sep 08 15:00:00.107134 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.107119    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:00.112234 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.112211    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-7f92a9-dockercfg-g95qr\""
Sep 08 15:00:00.112234 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.112227    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-7f92a9\""
Sep 08 15:00:00.125363 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.125333    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"]
Sep 08 15:00:00.134389 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.134364    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"]
Sep 08 15:00:00.137940 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.137919    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:00.159655 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.159635    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"]
Sep 08 15:00:00.161510 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.161491    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq"]
Sep 08 15:00:00.164899 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.164874    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd"]
Sep 08 15:00:00.165006 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.164987    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq"
Sep 08 15:00:00.167735 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.167719    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 15:00:00.167909 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.167896    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf"]
Sep 08 15:00:00.167997 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.167985    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd"
Sep 08 15:00:00.171037 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.171018    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf"
Sep 08 15:00:00.175559 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.175542    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq"
Sep 08 15:00:00.183259 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.183239    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pjpk\" (UniqueName: \"kubernetes.io/projected/87014c55-191a-445b-b455-7a86675dc5e6-kube-api-access-7pjpk\") pod \"payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r\" (UID: \"87014c55-191a-445b-b455-7a86675dc5e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:00.183408 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.183336    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a54c407-3118-49f5-84db-2daf9445e6b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n\" (UID: \"1a54c407-3118-49f5-84db-2daf9445e6b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:00.183408 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.183380    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87014c55-191a-445b-b455-7a86675dc5e6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r\" (UID: \"87014c55-191a-445b-b455-7a86675dc5e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:00.183535 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.183427    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v2js\" (UniqueName: \"kubernetes.io/projected/1a54c407-3118-49f5-84db-2daf9445e6b3-kube-api-access-8v2js\") pod \"payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n\" (UID: \"1a54c407-3118-49f5-84db-2daf9445e6b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:00.188926 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.188636    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd"
Sep 08 15:00:00.194235 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.192380    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t"]
Sep 08 15:00:00.201856 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.201729    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq"]
Sep 08 15:00:00.201856 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.199478    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t"
Sep 08 15:00:00.204168 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.204042    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf"]
Sep 08 15:00:00.208675 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.208652    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd"]
Sep 08 15:00:00.217403 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.217049    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t"]
Sep 08 15:00:00.231456 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.231427    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf"
Sep 08 15:00:00.244675 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.244622    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t"
Sep 08 15:00:00.285071 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.285026    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87014c55-191a-445b-b455-7a86675dc5e6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r\" (UID: \"87014c55-191a-445b-b455-7a86675dc5e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:00.285213 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.285100    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2js\" (UniqueName: \"kubernetes.io/projected/1a54c407-3118-49f5-84db-2daf9445e6b3-kube-api-access-8v2js\") pod \"payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n\" (UID: \"1a54c407-3118-49f5-84db-2daf9445e6b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:00.285213 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.285182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pjpk\" (UniqueName: \"kubernetes.io/projected/87014c55-191a-445b-b455-7a86675dc5e6-kube-api-access-7pjpk\") pod \"payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r\" (UID: \"87014c55-191a-445b-b455-7a86675dc5e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:00.285339 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.285290    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a54c407-3118-49f5-84db-2daf9445e6b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n\" (UID: \"1a54c407-3118-49f5-84db-2daf9445e6b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:00.287130 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.287098    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87014c55-191a-445b-b455-7a86675dc5e6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r\" (UID: \"87014c55-191a-445b-b455-7a86675dc5e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:00.287693 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.287514    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a54c407-3118-49f5-84db-2daf9445e6b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n\" (UID: \"1a54c407-3118-49f5-84db-2daf9445e6b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:00.301711 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.301603    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v2js\" (UniqueName: \"kubernetes.io/projected/1a54c407-3118-49f5-84db-2daf9445e6b3-kube-api-access-8v2js\") pod \"payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n\" (UID: \"1a54c407-3118-49f5-84db-2daf9445e6b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:00.303826 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.303795    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pjpk\" (UniqueName: \"kubernetes.io/projected/87014c55-191a-445b-b455-7a86675dc5e6-kube-api-access-7pjpk\") pod \"payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r\" (UID: \"87014c55-191a-445b-b455-7a86675dc5e6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:00.419993 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.419909    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:00.450406 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.450377    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:00.497333 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.497298    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq"]
Sep 08 15:00:00.506520 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.506268    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd"]
Sep 08 15:00:00.508968 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:00:00.508941    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7deb5039_5e5b_4c31_87f0_fb7be7047729.slice/crio-0c3af623a6f3ebe6af83dedb815e5fddcd9d076c7ebce17483c01123e0e7da11 WatchSource:0}: Error finding container 0c3af623a6f3ebe6af83dedb815e5fddcd9d076c7ebce17483c01123e0e7da11: Status 404 returned error can't find the container with id 0c3af623a6f3ebe6af83dedb815e5fddcd9d076c7ebce17483c01123e0e7da11
Sep 08 15:00:00.538345 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.538313    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf"]
Sep 08 15:00:00.555343 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.553993    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t"]
Sep 08 15:00:00.555343 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:00:00.554655    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod293503f7_eb71_4de1_bc58_d3dc25763d31.slice/crio-8014e749020125d5f203e0fa9cb2dc323d700ca9ecf4eed450de70c7751d8f0a WatchSource:0}: Error finding container 8014e749020125d5f203e0fa9cb2dc323d700ca9ecf4eed450de70c7751d8f0a: Status 404 returned error can't find the container with id 8014e749020125d5f203e0fa9cb2dc323d700ca9ecf4eed450de70c7751d8f0a
Sep 08 15:00:00.560371 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:00:00.560002    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05692f9b_624d_453a_b6d4_fed501485c8d.slice/crio-95d375236a4a9eb2e6c34f0561a69aaa1f8d66d8c35b845ae693b8c27c043fac WatchSource:0}: Error finding container 95d375236a4a9eb2e6c34f0561a69aaa1f8d66d8c35b845ae693b8c27c043fac: Status 404 returned error can't find the container with id 95d375236a4a9eb2e6c34f0561a69aaa1f8d66d8c35b845ae693b8c27c043fac
Sep 08 15:00:00.561044 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.561013    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd" event={"ID":"7deb5039-5e5b-4c31-87f0-fb7be7047729","Type":"ContainerStarted","Data":"0c3af623a6f3ebe6af83dedb815e5fddcd9d076c7ebce17483c01123e0e7da11"}
Sep 08 15:00:00.562616 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.562582    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq" event={"ID":"e0432e3a-a613-4fe1-b2c4-3bc17598281f","Type":"ContainerStarted","Data":"6be613027731a853841033c6003dde8d7c385d325e2b90139aa1554e9d8c80ce"}
Sep 08 15:00:00.702046 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.702023    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"]
Sep 08 15:00:00.741408 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:00.741385    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"]
Sep 08 15:00:00.776747 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:00:00.776703    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a54c407_3118_49f5_84db_2daf9445e6b3.slice/crio-49db2042ee908c0eb8dcf4a0d48cd3c325f1b3cde8d3f4d370198dc9ceabea8f WatchSource:0}: Error finding container 49db2042ee908c0eb8dcf4a0d48cd3c325f1b3cde8d3f4d370198dc9ceabea8f: Status 404 returned error can't find the container with id 49db2042ee908c0eb8dcf4a0d48cd3c325f1b3cde8d3f4d370198dc9ceabea8f
Sep 08 15:00:00.777402 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:00:00.777382    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87014c55_191a_445b_b455_7a86675dc5e6.slice/crio-6e13cc7fb568f195b63181a770def4e550c3177fa391e11a9f0ced65eaa6b300 WatchSource:0}: Error finding container 6e13cc7fb568f195b63181a770def4e550c3177fa391e11a9f0ced65eaa6b300: Status 404 returned error can't find the container with id 6e13cc7fb568f195b63181a770def4e550c3177fa391e11a9f0ced65eaa6b300
Sep 08 15:00:01.294523 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.294494    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:01.461342 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.461311    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:01.571621 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.571534    2583 generic.go:356] "Generic (PLEG): container finished" podID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerID="669f3bf2448877a8d27a2722f3c0daa2202318dce6aec7e64916a02a389714c5" exitCode=7
Sep 08 15:00:01.572080 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.571628    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd" event={"ID":"7deb5039-5e5b-4c31-87f0-fb7be7047729","Type":"ContainerDied","Data":"669f3bf2448877a8d27a2722f3c0daa2202318dce6aec7e64916a02a389714c5"}
Sep 08 15:00:01.572080 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.571790    2583 scope.go:117] "RemoveContainer" containerID="669f3bf2448877a8d27a2722f3c0daa2202318dce6aec7e64916a02a389714c5"
Sep 08 15:00:01.573540 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.573514    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf" event={"ID":"293503f7-eb71-4de1-bc58-d3dc25763d31","Type":"ContainerStarted","Data":"1a2f6a1234d760c83def8bd2db20a8f5e90da7326f1db77270a62ce6ad6bf7da"}
Sep 08 15:00:01.573684 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.573564    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf" event={"ID":"293503f7-eb71-4de1-bc58-d3dc25763d31","Type":"ContainerStarted","Data":"8014e749020125d5f203e0fa9cb2dc323d700ca9ecf4eed450de70c7751d8f0a"}
Sep 08 15:00:01.576747 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.575660    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n" event={"ID":"1a54c407-3118-49f5-84db-2daf9445e6b3","Type":"ContainerStarted","Data":"384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2"}
Sep 08 15:00:01.576747 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.576357    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n" event={"ID":"1a54c407-3118-49f5-84db-2daf9445e6b3","Type":"ContainerStarted","Data":"49db2042ee908c0eb8dcf4a0d48cd3c325f1b3cde8d3f4d370198dc9ceabea8f"}
Sep 08 15:00:01.576747 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.576391    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:01.578861 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.578768    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t" event={"ID":"05692f9b-624d-453a-b6d4-fed501485c8d","Type":"ContainerStarted","Data":"61ee4e20d87b61d8f3f5c936bbeb070c1cd15a3807a031102058fcb10e820d2e"}
Sep 08 15:00:01.578861 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.578813    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t" event={"ID":"05692f9b-624d-453a-b6d4-fed501485c8d","Type":"ContainerStarted","Data":"95d375236a4a9eb2e6c34f0561a69aaa1f8d66d8c35b845ae693b8c27c043fac"}
Sep 08 15:00:01.580938 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.580908    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r" event={"ID":"87014c55-191a-445b-b455-7a86675dc5e6","Type":"ContainerStarted","Data":"c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7"}
Sep 08 15:00:01.581045 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.580962    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r" event={"ID":"87014c55-191a-445b-b455-7a86675dc5e6","Type":"ContainerStarted","Data":"6e13cc7fb568f195b63181a770def4e550c3177fa391e11a9f0ced65eaa6b300"}
Sep 08 15:00:01.581045 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.580995    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:01.586576 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.586553    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq" event={"ID":"e0432e3a-a613-4fe1-b2c4-3bc17598281f","Type":"ContainerStarted","Data":"a9ed4b3a062388fdf2bfbf234bd2f8a7afaa804312a8bdb02a654893deaf48d7"}
Sep 08 15:00:01.606616 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.606565    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n" podStartSLOduration=1.6065516720000002 podStartE2EDuration="1.606551672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:00:01.6048058 +0000 UTC m=+1552.332622815" watchObservedRunningTime="2026-09-08 15:00:01.606551672 +0000 UTC m=+1552.334368686"
Sep 08 15:00:01.622789 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.622747    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf" podStartSLOduration=1.622731009 podStartE2EDuration="1.622731009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:00:01.620715781 +0000 UTC m=+1552.348532796" watchObservedRunningTime="2026-09-08 15:00:01.622731009 +0000 UTC m=+1552.350548040"
Sep 08 15:00:01.639583 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.639544    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq" podStartSLOduration=1.639530953 podStartE2EDuration="1.639530953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:00:01.637892558 +0000 UTC m=+1552.365709574" watchObservedRunningTime="2026-09-08 15:00:01.639530953 +0000 UTC m=+1552.367347966"
Sep 08 15:00:01.660796 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.660744    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r" podStartSLOduration=1.660730444 podStartE2EDuration="1.660730444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:00:01.658037766 +0000 UTC m=+1552.385854781" watchObservedRunningTime="2026-09-08 15:00:01.660730444 +0000 UTC m=+1552.388547457"
Sep 08 15:00:01.675018 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:01.674970    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t" podStartSLOduration=1.674957977 podStartE2EDuration="1.674957977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:00:01.673337126 +0000 UTC m=+1552.401154141" watchObservedRunningTime="2026-09-08 15:00:01.674957977 +0000 UTC m=+1552.402774998"
Sep 08 15:00:02.594050 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:02.594009    2583 generic.go:356] "Generic (PLEG): container finished" podID="e0432e3a-a613-4fe1-b2c4-3bc17598281f" containerID="a9ed4b3a062388fdf2bfbf234bd2f8a7afaa804312a8bdb02a654893deaf48d7" exitCode=0
Sep 08 15:00:02.594546 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:02.594049    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq" event={"ID":"e0432e3a-a613-4fe1-b2c4-3bc17598281f","Type":"ContainerDied","Data":"a9ed4b3a062388fdf2bfbf234bd2f8a7afaa804312a8bdb02a654893deaf48d7"}
Sep 08 15:00:02.596269 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:02.596246    2583 generic.go:356] "Generic (PLEG): container finished" podID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerID="b3fd04e181913e696b286451a468aaca6b38b5b547077f03582368aa76e0d92f" exitCode=7
Sep 08 15:00:02.596404 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:02.596333    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd" event={"ID":"7deb5039-5e5b-4c31-87f0-fb7be7047729","Type":"ContainerDied","Data":"b3fd04e181913e696b286451a468aaca6b38b5b547077f03582368aa76e0d92f"}
Sep 08 15:00:02.596404 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:02.596393    2583 scope.go:117] "RemoveContainer" containerID="669f3bf2448877a8d27a2722f3c0daa2202318dce6aec7e64916a02a389714c5"
Sep 08 15:00:02.596573 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:02.596544    2583 scope.go:117] "RemoveContainer" containerID="b3fd04e181913e696b286451a468aaca6b38b5b547077f03582368aa76e0d92f"
Sep 08 15:00:02.596877 ip-10-0-137-6 kubenswrapper[2583]: E0908 15:00:02.596852    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd_odh-ai-gateway-infra(7deb5039-5e5b-4c31-87f0-fb7be7047729)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729"
Sep 08 15:00:02.598005 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:02.597988    2583 generic.go:356] "Generic (PLEG): container finished" podID="293503f7-eb71-4de1-bc58-d3dc25763d31" containerID="1a2f6a1234d760c83def8bd2db20a8f5e90da7326f1db77270a62ce6ad6bf7da" exitCode=0
Sep 08 15:00:02.598099 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:02.598074    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf" event={"ID":"293503f7-eb71-4de1-bc58-d3dc25763d31","Type":"ContainerDied","Data":"1a2f6a1234d760c83def8bd2db20a8f5e90da7326f1db77270a62ce6ad6bf7da"}
Sep 08 15:00:02.599607 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:02.599587    2583 generic.go:356] "Generic (PLEG): container finished" podID="05692f9b-624d-453a-b6d4-fed501485c8d" containerID="61ee4e20d87b61d8f3f5c936bbeb070c1cd15a3807a031102058fcb10e820d2e" exitCode=0
Sep 08 15:00:02.599709 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:02.599654    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t" event={"ID":"05692f9b-624d-453a-b6d4-fed501485c8d","Type":"ContainerDied","Data":"61ee4e20d87b61d8f3f5c936bbeb070c1cd15a3807a031102058fcb10e820d2e"}
Sep 08 15:00:03.607299 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:03.607251    2583 scope.go:117] "RemoveContainer" containerID="b3fd04e181913e696b286451a468aaca6b38b5b547077f03582368aa76e0d92f"
Sep 08 15:00:03.607755 ip-10-0-137-6 kubenswrapper[2583]: E0908 15:00:03.607541    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd_odh-ai-gateway-infra(7deb5039-5e5b-4c31-87f0-fb7be7047729)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729"
Sep 08 15:00:03.755794 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:03.755758    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf"
Sep 08 15:00:03.788928 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:03.788898    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq"
Sep 08 15:00:03.832585 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:03.832563    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t"
Sep 08 15:00:04.613703 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:04.613669    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq"
Sep 08 15:00:04.613703 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:04.613703    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814660-5m5pq" event={"ID":"e0432e3a-a613-4fe1-b2c4-3bc17598281f","Type":"ContainerDied","Data":"6be613027731a853841033c6003dde8d7c385d325e2b90139aa1554e9d8c80ce"}
Sep 08 15:00:04.614207 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:04.613746    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6be613027731a853841033c6003dde8d7c385d325e2b90139aa1554e9d8c80ce"
Sep 08 15:00:04.615267 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:04.615248    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf"
Sep 08 15:00:04.615401 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:04.615275    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf" event={"ID":"293503f7-eb71-4de1-bc58-d3dc25763d31","Type":"ContainerDied","Data":"8014e749020125d5f203e0fa9cb2dc323d700ca9ecf4eed450de70c7751d8f0a"}
Sep 08 15:00:04.615401 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:04.615314    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8014e749020125d5f203e0fa9cb2dc323d700ca9ecf4eed450de70c7751d8f0a"
Sep 08 15:00:04.616758 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:04.616735    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t" event={"ID":"05692f9b-624d-453a-b6d4-fed501485c8d","Type":"ContainerDied","Data":"95d375236a4a9eb2e6c34f0561a69aaa1f8d66d8c35b845ae693b8c27c043fac"}
Sep 08 15:00:04.616900 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:04.616776    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95d375236a4a9eb2e6c34f0561a69aaa1f8d66d8c35b845ae693b8c27c043fac"
Sep 08 15:00:04.616900 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:04.616744    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t"
Sep 08 15:00:10.020825 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:10.020789    2583 scope.go:117] "RemoveContainer" containerID="c715f01caa3fd780c9c5db37656edc3d83f4b36df63551971d7862c111520a59"
Sep 08 15:00:12.602414 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:12.602385    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:12.602849 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:12.602693    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:15.787469 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:15.787439    2583 scope.go:117] "RemoveContainer" containerID="b3fd04e181913e696b286451a468aaca6b38b5b547077f03582368aa76e0d92f"
Sep 08 15:00:16.683104 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:16.683067    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd" event={"ID":"7deb5039-5e5b-4c31-87f0-fb7be7047729","Type":"ContainerStarted","Data":"441aecb4a3b14e9e1ea7ad898d1aef2d46ba17d2ccac744c871f99493944f9e2"}
Sep 08 15:00:16.698570 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:16.698529    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd" podStartSLOduration=16.698515696 podStartE2EDuration="16.698515696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:00:16.697600924 +0000 UTC m=+1567.425417938" watchObservedRunningTime="2026-09-08 15:00:16.698515696 +0000 UTC m=+1567.426332710"
Sep 08 15:00:17.690623 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:17.690588    2583 generic.go:356] "Generic (PLEG): container finished" podID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerID="441aecb4a3b14e9e1ea7ad898d1aef2d46ba17d2ccac744c871f99493944f9e2" exitCode=0
Sep 08 15:00:17.691058 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:17.690665    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd" event={"ID":"7deb5039-5e5b-4c31-87f0-fb7be7047729","Type":"ContainerDied","Data":"441aecb4a3b14e9e1ea7ad898d1aef2d46ba17d2ccac744c871f99493944f9e2"}
Sep 08 15:00:17.691058 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:17.690729    2583 scope.go:117] "RemoveContainer" containerID="b3fd04e181913e696b286451a468aaca6b38b5b547077f03582368aa76e0d92f"
Sep 08 15:00:17.708731 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:17.708706    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd"]
Sep 08 15:00:18.827807 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:18.827781    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd"
Sep 08 15:00:19.704445 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:19.704412    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd" event={"ID":"7deb5039-5e5b-4c31-87f0-fb7be7047729","Type":"ContainerDied","Data":"0c3af623a6f3ebe6af83dedb815e5fddcd9d076c7ebce17483c01123e0e7da11"}
Sep 08 15:00:19.704445 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:19.704430    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd"
Sep 08 15:00:19.704675 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:19.704457    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c3af623a6f3ebe6af83dedb815e5fddcd9d076c7ebce17483c01123e0e7da11"
Sep 08 15:00:19.735897 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:19.735868    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd"]
Sep 08 15:00:19.740922 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:19.740896    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-cleanup-7f92a9-29814660-ffjrd"]
Sep 08 15:00:19.794373 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:19.794334    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729" path="/var/lib/kubelet/pods/7deb5039-5e5b-4c31-87f0-fb7be7047729/volumes"
Sep 08 15:00:21.190040 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:21.190003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:23.333126 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:23.333070    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:24.063942 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.063905    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"]
Sep 08 15:00:24.064327 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064314    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05692f9b-624d-453a-b6d4-fed501485c8d" containerName="cleanup"
Sep 08 15:00:24.064371 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064328    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="05692f9b-624d-453a-b6d4-fed501485c8d" containerName="cleanup"
Sep 08 15:00:24.064371 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064338    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerName="cleanup"
Sep 08 15:00:24.064371 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064344    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerName="cleanup"
Sep 08 15:00:24.064371 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064358    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="293503f7-eb71-4de1-bc58-d3dc25763d31" containerName="cleanup"
Sep 08 15:00:24.064371 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064364    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="293503f7-eb71-4de1-bc58-d3dc25763d31" containerName="cleanup"
Sep 08 15:00:24.064371 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064371    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0432e3a-a613-4fe1-b2c4-3bc17598281f" containerName="cleanup"
Sep 08 15:00:24.064551 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064377    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0432e3a-a613-4fe1-b2c4-3bc17598281f" containerName="cleanup"
Sep 08 15:00:24.064551 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064388    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerName="cleanup"
Sep 08 15:00:24.064551 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064394    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerName="cleanup"
Sep 08 15:00:24.064551 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064402    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerName="cleanup"
Sep 08 15:00:24.064551 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064408    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerName="cleanup"
Sep 08 15:00:24.064551 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064469    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0432e3a-a613-4fe1-b2c4-3bc17598281f" containerName="cleanup"
Sep 08 15:00:24.064551 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064475    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="293503f7-eb71-4de1-bc58-d3dc25763d31" containerName="cleanup"
Sep 08 15:00:24.064551 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064482    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerName="cleanup"
Sep 08 15:00:24.064551 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064489    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="05692f9b-624d-453a-b6d4-fed501485c8d" containerName="cleanup"
Sep 08 15:00:24.064551 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.064496    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerName="cleanup"
Sep 08 15:00:24.067517 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.067501    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"
Sep 08 15:00:24.070244 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.070223    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:00:24.070351 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.070244    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 15:00:24.075379 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.075350    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"]
Sep 08 15:00:24.120177 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.120139    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/163b1f57-01ff-4432-ac26-b23c2da3feb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl\" (UID: \"163b1f57-01ff-4432-ac26-b23c2da3feb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"
Sep 08 15:00:24.221474 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.221437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/163b1f57-01ff-4432-ac26-b23c2da3feb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl\" (UID: \"163b1f57-01ff-4432-ac26-b23c2da3feb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"
Sep 08 15:00:24.222035 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.222019    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/163b1f57-01ff-4432-ac26-b23c2da3feb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl\" (UID: \"163b1f57-01ff-4432-ac26-b23c2da3feb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"
Sep 08 15:00:24.380040 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.379947    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"
Sep 08 15:00:24.652169 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.652145    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"]
Sep 08 15:00:24.654221 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:00:24.654194    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163b1f57_01ff_4432_ac26_b23c2da3feb1.slice/crio-84573c882ab29218d081b97872dc73c3dbf08b81a0fc5868b0b1f694069e96af WatchSource:0}: Error finding container 84573c882ab29218d081b97872dc73c3dbf08b81a0fc5868b0b1f694069e96af: Status 404 returned error can't find the container with id 84573c882ab29218d081b97872dc73c3dbf08b81a0fc5868b0b1f694069e96af
Sep 08 15:00:24.731587 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:24.731561    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl" event={"ID":"163b1f57-01ff-4432-ac26-b23c2da3feb1","Type":"ContainerStarted","Data":"84573c882ab29218d081b97872dc73c3dbf08b81a0fc5868b0b1f694069e96af"}
Sep 08 15:00:25.738635 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:25.738600    2583 generic.go:356] "Generic (PLEG): container finished" podID="163b1f57-01ff-4432-ac26-b23c2da3feb1" containerID="332ff815c8ca4a2319e1e9c2d565261793b30340d524fa9d10f2d3e3e8a9f260" exitCode=0
Sep 08 15:00:25.739031 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:25.738689    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl" event={"ID":"163b1f57-01ff-4432-ac26-b23c2da3feb1","Type":"ContainerDied","Data":"332ff815c8ca4a2319e1e9c2d565261793b30340d524fa9d10f2d3e3e8a9f260"}
Sep 08 15:00:26.878533 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:26.878511    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"
Sep 08 15:00:26.947928 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:26.947903    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/163b1f57-01ff-4432-ac26-b23c2da3feb1-maas-api-service-ca\") pod \"163b1f57-01ff-4432-ac26-b23c2da3feb1\" (UID: \"163b1f57-01ff-4432-ac26-b23c2da3feb1\") "
Sep 08 15:00:26.948230 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:26.948208    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/163b1f57-01ff-4432-ac26-b23c2da3feb1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "163b1f57-01ff-4432-ac26-b23c2da3feb1" (UID: "163b1f57-01ff-4432-ac26-b23c2da3feb1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:00:27.049481 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:27.049396    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/163b1f57-01ff-4432-ac26-b23c2da3feb1-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:00:27.750919 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:27.750882    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl" event={"ID":"163b1f57-01ff-4432-ac26-b23c2da3feb1","Type":"ContainerDied","Data":"84573c882ab29218d081b97872dc73c3dbf08b81a0fc5868b0b1f694069e96af"}
Sep 08 15:00:27.750919 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:27.750917    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"
Sep 08 15:00:27.751143 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:27.750929    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84573c882ab29218d081b97872dc73c3dbf08b81a0fc5868b0b1f694069e96af"
Sep 08 15:00:29.962277 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:29.962233    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"]
Sep 08 15:00:29.962737 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:29.962488    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r" podUID="87014c55-191a-445b-b455-7a86675dc5e6" containerName="payload-processing" containerID="cri-o://c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7" gracePeriod=30
Sep 08 15:00:29.968935 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:29.967774    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"]
Sep 08 15:00:29.969497 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:29.969450    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n" podUID="1a54c407-3118-49f5-84db-2daf9445e6b3" containerName="payload-pre-processing" containerID="cri-o://384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2" gracePeriod=30
Sep 08 15:00:30.030375 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.030343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:30.104132 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.104088    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"]
Sep 08 15:00:30.107589 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.107562    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7f92a9-7jmdl"]
Sep 08 15:00:30.251369 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.251341    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:30.254307 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.254287    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:30.279787 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.279754    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pjpk\" (UniqueName: \"kubernetes.io/projected/87014c55-191a-445b-b455-7a86675dc5e6-kube-api-access-7pjpk\") pod \"87014c55-191a-445b-b455-7a86675dc5e6\" (UID: \"87014c55-191a-445b-b455-7a86675dc5e6\") "
Sep 08 15:00:30.279949 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.279858    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a54c407-3118-49f5-84db-2daf9445e6b3-plugins-config-volume\") pod \"1a54c407-3118-49f5-84db-2daf9445e6b3\" (UID: \"1a54c407-3118-49f5-84db-2daf9445e6b3\") "
Sep 08 15:00:30.279993 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.279956    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87014c55-191a-445b-b455-7a86675dc5e6-plugins-config-volume\") pod \"87014c55-191a-445b-b455-7a86675dc5e6\" (UID: \"87014c55-191a-445b-b455-7a86675dc5e6\") "
Sep 08 15:00:30.280048 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.280005    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v2js\" (UniqueName: \"kubernetes.io/projected/1a54c407-3118-49f5-84db-2daf9445e6b3-kube-api-access-8v2js\") pod \"1a54c407-3118-49f5-84db-2daf9445e6b3\" (UID: \"1a54c407-3118-49f5-84db-2daf9445e6b3\") "
Sep 08 15:00:30.280251 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.280223    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a54c407-3118-49f5-84db-2daf9445e6b3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a54c407-3118-49f5-84db-2daf9445e6b3" (UID: "1a54c407-3118-49f5-84db-2daf9445e6b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:00:30.280385 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.280282    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87014c55-191a-445b-b455-7a86675dc5e6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "87014c55-191a-445b-b455-7a86675dc5e6" (UID: "87014c55-191a-445b-b455-7a86675dc5e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:00:30.282259 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.282232    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87014c55-191a-445b-b455-7a86675dc5e6-kube-api-access-7pjpk" (OuterVolumeSpecName: "kube-api-access-7pjpk") pod "87014c55-191a-445b-b455-7a86675dc5e6" (UID: "87014c55-191a-445b-b455-7a86675dc5e6"). InnerVolumeSpecName "kube-api-access-7pjpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:00:30.282354 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.282235    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a54c407-3118-49f5-84db-2daf9445e6b3-kube-api-access-8v2js" (OuterVolumeSpecName: "kube-api-access-8v2js") pod "1a54c407-3118-49f5-84db-2daf9445e6b3" (UID: "1a54c407-3118-49f5-84db-2daf9445e6b3"). InnerVolumeSpecName "kube-api-access-8v2js". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:00:30.381273 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.381239    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a54c407-3118-49f5-84db-2daf9445e6b3-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:00:30.381273 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.381271    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87014c55-191a-445b-b455-7a86675dc5e6-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:00:30.381273 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.381282    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v2js\" (UniqueName: \"kubernetes.io/projected/1a54c407-3118-49f5-84db-2daf9445e6b3-kube-api-access-8v2js\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:00:30.381511 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.381291    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pjpk\" (UniqueName: \"kubernetes.io/projected/87014c55-191a-445b-b455-7a86675dc5e6-kube-api-access-7pjpk\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:00:30.767908 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.767873    2583 generic.go:356] "Generic (PLEG): container finished" podID="1a54c407-3118-49f5-84db-2daf9445e6b3" containerID="384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2" exitCode=0
Sep 08 15:00:30.768101 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.767910    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"
Sep 08 15:00:30.768101 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.767962    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n" event={"ID":"1a54c407-3118-49f5-84db-2daf9445e6b3","Type":"ContainerDied","Data":"384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2"}
Sep 08 15:00:30.768101 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.768020    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n" event={"ID":"1a54c407-3118-49f5-84db-2daf9445e6b3","Type":"ContainerDied","Data":"49db2042ee908c0eb8dcf4a0d48cd3c325f1b3cde8d3f4d370198dc9ceabea8f"}
Sep 08 15:00:30.768101 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.768053    2583 scope.go:117] "RemoveContainer" containerID="384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2"
Sep 08 15:00:30.769369 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.769351    2583 generic.go:356] "Generic (PLEG): container finished" podID="87014c55-191a-445b-b455-7a86675dc5e6" containerID="c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7" exitCode=0
Sep 08 15:00:30.769448 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.769412    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"
Sep 08 15:00:30.769512 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.769411    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r" event={"ID":"87014c55-191a-445b-b455-7a86675dc5e6","Type":"ContainerDied","Data":"c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7"}
Sep 08 15:00:30.769567 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.769551    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r" event={"ID":"87014c55-191a-445b-b455-7a86675dc5e6","Type":"ContainerDied","Data":"6e13cc7fb568f195b63181a770def4e550c3177fa391e11a9f0ced65eaa6b300"}
Sep 08 15:00:30.778625 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.778600    2583 scope.go:117] "RemoveContainer" containerID="384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2"
Sep 08 15:00:30.779035 ip-10-0-137-6 kubenswrapper[2583]: E0908 15:00:30.779009    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2\": container with ID starting with 384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2 not found: ID does not exist" containerID="384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2"
Sep 08 15:00:30.779120 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.779046    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2"} err="failed to get container status \"384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2\": rpc error: code = NotFound desc = could not find container \"384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2\": container with ID starting with 384e90775053565e38e7cc6f989e305a3dc6e1943e8eef702092081345a4dea2 not found: ID does not exist"
Sep 08 15:00:30.779120 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.779071    2583 scope.go:117] "RemoveContainer" containerID="c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7"
Sep 08 15:00:30.786965 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.786943    2583 scope.go:117] "RemoveContainer" containerID="c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7"
Sep 08 15:00:30.787210 ip-10-0-137-6 kubenswrapper[2583]: E0908 15:00:30.787193    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7\": container with ID starting with c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7 not found: ID does not exist" containerID="c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7"
Sep 08 15:00:30.787273 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.787217    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7"} err="failed to get container status \"c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7\": rpc error: code = NotFound desc = could not find container \"c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7\": container with ID starting with c1c0996cfb2404809d1c1a7beb1757ec97f0acb3e94be2b73c0a4885132951d7 not found: ID does not exist"
Sep 08 15:00:30.790782 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.790760    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"]
Sep 08 15:00:30.795995 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.795975    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7f92a9-579784589d-q4j2n"]
Sep 08 15:00:30.807725 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.807701    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"]
Sep 08 15:00:30.810431 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:30.810412    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7f92a9-6d5f5c7b99-r962r"]
Sep 08 15:00:31.793805 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:31.793774    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="163b1f57-01ff-4432-ac26-b23c2da3feb1" path="/var/lib/kubelet/pods/163b1f57-01ff-4432-ac26-b23c2da3feb1/volumes"
Sep 08 15:00:31.794185 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:31.794120    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a54c407-3118-49f5-84db-2daf9445e6b3" path="/var/lib/kubelet/pods/1a54c407-3118-49f5-84db-2daf9445e6b3/volumes"
Sep 08 15:00:31.794425 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:31.794411    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87014c55-191a-445b-b455-7a86675dc5e6" path="/var/lib/kubelet/pods/87014c55-191a-445b-b455-7a86675dc5e6/volumes"
Sep 08 15:00:32.070872 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:32.070761    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:32.171081 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:32.171052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:36.272401 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:36.272363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:36.372872 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:36.372819    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:36.546732 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:36.546654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:40.667920 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.667883    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"]
Sep 08 15:00:40.668299 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.668287    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="163b1f57-01ff-4432-ac26-b23c2da3feb1" containerName="revoke-keys"
Sep 08 15:00:40.668340 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.668300    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="163b1f57-01ff-4432-ac26-b23c2da3feb1" containerName="revoke-keys"
Sep 08 15:00:40.668340 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.668313    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a54c407-3118-49f5-84db-2daf9445e6b3" containerName="payload-pre-processing"
Sep 08 15:00:40.668340 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.668319    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a54c407-3118-49f5-84db-2daf9445e6b3" containerName="payload-pre-processing"
Sep 08 15:00:40.668340 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.668325    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87014c55-191a-445b-b455-7a86675dc5e6" containerName="payload-processing"
Sep 08 15:00:40.668340 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.668332    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="87014c55-191a-445b-b455-7a86675dc5e6" containerName="payload-processing"
Sep 08 15:00:40.668492 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.668399    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="163b1f57-01ff-4432-ac26-b23c2da3feb1" containerName="revoke-keys"
Sep 08 15:00:40.668492 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.668408    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a54c407-3118-49f5-84db-2daf9445e6b3" containerName="payload-pre-processing"
Sep 08 15:00:40.668492 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.668413    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7deb5039-5e5b-4c31-87f0-fb7be7047729" containerName="cleanup"
Sep 08 15:00:40.668492 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.668422    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="87014c55-191a-445b-b455-7a86675dc5e6" containerName="payload-processing"
Sep 08 15:00:40.670816 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.670798    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"
Sep 08 15:00:40.673572 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.673548    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:00:40.673692 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.673560    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 15:00:40.681053 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.681030    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"]
Sep 08 15:00:40.783959 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.783927    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15361e94-f1e1-4811-81c5-5f5cbe5c92bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4\" (UID: \"15361e94-f1e1-4811-81c5-5f5cbe5c92bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"
Sep 08 15:00:40.803195 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.803168    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:40.884672 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.884636    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15361e94-f1e1-4811-81c5-5f5cbe5c92bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4\" (UID: \"15361e94-f1e1-4811-81c5-5f5cbe5c92bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"
Sep 08 15:00:40.885394 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.885374    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15361e94-f1e1-4811-81c5-5f5cbe5c92bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4\" (UID: \"15361e94-f1e1-4811-81c5-5f5cbe5c92bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"
Sep 08 15:00:40.983012 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:40.982992    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"
Sep 08 15:00:41.254722 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:41.254699    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"]
Sep 08 15:00:41.256500 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:00:41.256471    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15361e94_f1e1_4811_81c5_5f5cbe5c92bd.slice/crio-eccf02263c3693d8c4fbcc80f0e21749e11579555778b06de992216280353e31 WatchSource:0}: Error finding container eccf02263c3693d8c4fbcc80f0e21749e11579555778b06de992216280353e31: Status 404 returned error can't find the container with id eccf02263c3693d8c4fbcc80f0e21749e11579555778b06de992216280353e31
Sep 08 15:00:41.832463 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:41.832373    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4" event={"ID":"15361e94-f1e1-4811-81c5-5f5cbe5c92bd","Type":"ContainerStarted","Data":"f1ae5ac4ee705e9164d89bc263b70e0ee4eebc3351bd4ee591242d00187942af"}
Sep 08 15:00:41.832463 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:41.832435    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4" event={"ID":"15361e94-f1e1-4811-81c5-5f5cbe5c92bd","Type":"ContainerStarted","Data":"eccf02263c3693d8c4fbcc80f0e21749e11579555778b06de992216280353e31"}
Sep 08 15:00:41.849869 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:41.849803    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4" podStartSLOduration=1.849787233 podStartE2EDuration="1.849787233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:00:41.848593032 +0000 UTC m=+1592.576410048" watchObservedRunningTime="2026-09-08 15:00:41.849787233 +0000 UTC m=+1592.577604247"
Sep 08 15:00:42.838772 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:42.838736    2583 generic.go:356] "Generic (PLEG): container finished" podID="15361e94-f1e1-4811-81c5-5f5cbe5c92bd" containerID="f1ae5ac4ee705e9164d89bc263b70e0ee4eebc3351bd4ee591242d00187942af" exitCode=0
Sep 08 15:00:42.839206 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:42.838776    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4" event={"ID":"15361e94-f1e1-4811-81c5-5f5cbe5c92bd","Type":"ContainerDied","Data":"f1ae5ac4ee705e9164d89bc263b70e0ee4eebc3351bd4ee591242d00187942af"}
Sep 08 15:00:43.973294 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:43.973268    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"
Sep 08 15:00:44.114462 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:44.114369    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15361e94-f1e1-4811-81c5-5f5cbe5c92bd-maas-api-service-ca\") pod \"15361e94-f1e1-4811-81c5-5f5cbe5c92bd\" (UID: \"15361e94-f1e1-4811-81c5-5f5cbe5c92bd\") "
Sep 08 15:00:44.114784 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:44.114751    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15361e94-f1e1-4811-81c5-5f5cbe5c92bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15361e94-f1e1-4811-81c5-5f5cbe5c92bd" (UID: "15361e94-f1e1-4811-81c5-5f5cbe5c92bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:00:44.215362 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:44.215327    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15361e94-f1e1-4811-81c5-5f5cbe5c92bd-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:00:44.850405 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:44.850370    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"
Sep 08 15:00:44.850554 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:44.850410    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4" event={"ID":"15361e94-f1e1-4811-81c5-5f5cbe5c92bd","Type":"ContainerDied","Data":"eccf02263c3693d8c4fbcc80f0e21749e11579555778b06de992216280353e31"}
Sep 08 15:00:44.850554 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:44.850459    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eccf02263c3693d8c4fbcc80f0e21749e11579555778b06de992216280353e31"
Sep 08 15:00:56.100569 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:56.100529    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf"]
Sep 08 15:00:56.103116 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:56.103094    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-28eb2c-29814660-wz2tf"]
Sep 08 15:00:56.118421 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:56.118382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:56.216163 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:56.216123    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:56.290219 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:56.290178    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"]
Sep 08 15:00:56.293645 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:56.293619    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-28eb2c-xj6z4"]
Sep 08 15:00:56.293998 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:56.293975    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:56.355612 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:56.355539    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:56.715043 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:56.714958    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:00:57.793407 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:57.793372    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15361e94-f1e1-4811-81c5-5f5cbe5c92bd" path="/var/lib/kubelet/pods/15361e94-f1e1-4811-81c5-5f5cbe5c92bd/volumes"
Sep 08 15:00:57.793771 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:00:57.793639    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="293503f7-eb71-4de1-bc58-d3dc25763d31" path="/var/lib/kubelet/pods/293503f7-eb71-4de1-bc58-d3dc25763d31/volumes"
Sep 08 15:01:01.696524 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:01.696441    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"]
Sep 08 15:01:01.792414 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:01.792373    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:01:06.689875 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:06.689811    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:01:16.716790 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:16.716730    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf" podUID="1781cc54-80a5-48c0-9bfb-efd5515ef928" containerName="main" containerID="cri-o://f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c" gracePeriod=45
Sep 08 15:01:16.966655 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:16.966626    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 15:01:17.009410 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.009376    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-home\") pod \"1781cc54-80a5-48c0-9bfb-efd5515ef928\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") "
Sep 08 15:01:17.009410 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.009427    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-dshm\") pod \"1781cc54-80a5-48c0-9bfb-efd5515ef928\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") "
Sep 08 15:01:17.009694 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.009491    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmcn7\" (UniqueName: \"kubernetes.io/projected/1781cc54-80a5-48c0-9bfb-efd5515ef928-kube-api-access-cmcn7\") pod \"1781cc54-80a5-48c0-9bfb-efd5515ef928\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") "
Sep 08 15:01:17.009694 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.009541    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-tmp-dir\") pod \"1781cc54-80a5-48c0-9bfb-efd5515ef928\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") "
Sep 08 15:01:17.009694 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.009570    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-model-cache\") pod \"1781cc54-80a5-48c0-9bfb-efd5515ef928\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") "
Sep 08 15:01:17.009694 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.009603    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1781cc54-80a5-48c0-9bfb-efd5515ef928-tls-certs\") pod \"1781cc54-80a5-48c0-9bfb-efd5515ef928\" (UID: \"1781cc54-80a5-48c0-9bfb-efd5515ef928\") "
Sep 08 15:01:17.009968 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.009686    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-home" (OuterVolumeSpecName: "home") pod "1781cc54-80a5-48c0-9bfb-efd5515ef928" (UID: "1781cc54-80a5-48c0-9bfb-efd5515ef928"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:01:17.009968 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.009932    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1781cc54-80a5-48c0-9bfb-efd5515ef928" (UID: "1781cc54-80a5-48c0-9bfb-efd5515ef928"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:01:17.009968 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.009952    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:01:17.009968 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.009964    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-model-cache" (OuterVolumeSpecName: "model-cache") pod "1781cc54-80a5-48c0-9bfb-efd5515ef928" (UID: "1781cc54-80a5-48c0-9bfb-efd5515ef928"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:01:17.011966 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.011915    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-dshm" (OuterVolumeSpecName: "dshm") pod "1781cc54-80a5-48c0-9bfb-efd5515ef928" (UID: "1781cc54-80a5-48c0-9bfb-efd5515ef928"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:01:17.012106 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.012001    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1781cc54-80a5-48c0-9bfb-efd5515ef928-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1781cc54-80a5-48c0-9bfb-efd5515ef928" (UID: "1781cc54-80a5-48c0-9bfb-efd5515ef928"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:01:17.012256 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.012233    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1781cc54-80a5-48c0-9bfb-efd5515ef928-kube-api-access-cmcn7" (OuterVolumeSpecName: "kube-api-access-cmcn7") pod "1781cc54-80a5-48c0-9bfb-efd5515ef928" (UID: "1781cc54-80a5-48c0-9bfb-efd5515ef928"). InnerVolumeSpecName "kube-api-access-cmcn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:01:17.020430 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.020404    2583 generic.go:356] "Generic (PLEG): container finished" podID="1781cc54-80a5-48c0-9bfb-efd5515ef928" containerID="f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c" exitCode=0
Sep 08 15:01:17.020541 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.020465    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"
Sep 08 15:01:17.020541 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.020484    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf" event={"ID":"1781cc54-80a5-48c0-9bfb-efd5515ef928","Type":"ContainerDied","Data":"f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c"}
Sep 08 15:01:17.020541 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.020530    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf" event={"ID":"1781cc54-80a5-48c0-9bfb-efd5515ef928","Type":"ContainerDied","Data":"d1879935b04f2a8def26dfd378f458dfa4ad14e877847fc45c83ab59729c8c69"}
Sep 08 15:01:17.020680 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.020546    2583 scope.go:117] "RemoveContainer" containerID="f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c"
Sep 08 15:01:17.034472 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.034454    2583 scope.go:117] "RemoveContainer" containerID="f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c"
Sep 08 15:01:17.034738 ip-10-0-137-6 kubenswrapper[2583]: E0908 15:01:17.034721    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c\": container with ID starting with f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c not found: ID does not exist" containerID="f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c"
Sep 08 15:01:17.034790 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.034746    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c"} err="failed to get container status \"f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c\": rpc error: code = NotFound desc = could not find container \"f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c\": container with ID starting with f655675d7c8acc3c0663a567c076ae7889ccac437ad48ea9d394db28cc7d595c not found: ID does not exist"
Sep 08 15:01:17.048382 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.048355    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"]
Sep 08 15:01:17.050337 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.050317    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-28eb2c/ipp-route-28eb2c-kserve-5fccc44467-vq9wf"]
Sep 08 15:01:17.111115 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.111092    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:01:17.111115 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.111115    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:01:17.111241 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.111122    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1781cc54-80a5-48c0-9bfb-efd5515ef928-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:01:17.111241 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.111129    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1781cc54-80a5-48c0-9bfb-efd5515ef928-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:01:17.111241 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.111137    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmcn7\" (UniqueName: \"kubernetes.io/projected/1781cc54-80a5-48c0-9bfb-efd5515ef928-kube-api-access-cmcn7\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:01:17.793720 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:17.793688    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1781cc54-80a5-48c0-9bfb-efd5515ef928" path="/var/lib/kubelet/pods/1781cc54-80a5-48c0-9bfb-efd5515ef928/volumes"
Sep 08 15:01:34.537327 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:34.537289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:01:38.243232 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:38.243197    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:01:39.145156 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:39.145117    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:01:54.400546 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:54.400506    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:01:54.475374 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:54.475343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:01:54.728441 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:54.728410    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:01:55.072965 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.072885    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"]
Sep 08 15:01:55.073261 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.073249    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15361e94-f1e1-4811-81c5-5f5cbe5c92bd" containerName="revoke-keys"
Sep 08 15:01:55.073306 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.073262    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="15361e94-f1e1-4811-81c5-5f5cbe5c92bd" containerName="revoke-keys"
Sep 08 15:01:55.073306 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.073288    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1781cc54-80a5-48c0-9bfb-efd5515ef928" containerName="main"
Sep 08 15:01:55.073306 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.073293    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1781cc54-80a5-48c0-9bfb-efd5515ef928" containerName="main"
Sep 08 15:01:55.073441 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.073365    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="15361e94-f1e1-4811-81c5-5f5cbe5c92bd" containerName="revoke-keys"
Sep 08 15:01:55.073441 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.073377    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1781cc54-80a5-48c0-9bfb-efd5515ef928" containerName="main"
Sep 08 15:01:55.076416 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.076398    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"
Sep 08 15:01:55.079316 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.079297    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:01:55.079443 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.079364    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 15:01:55.086563 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.086529    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"]
Sep 08 15:01:55.150415 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.150377    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa04c39-0899-40d3-809b-5a1d03396342-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh\" (UID: \"8fa04c39-0899-40d3-809b-5a1d03396342\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"
Sep 08 15:01:55.190715 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.190685    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:01:55.251169 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.251139    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa04c39-0899-40d3-809b-5a1d03396342-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh\" (UID: \"8fa04c39-0899-40d3-809b-5a1d03396342\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"
Sep 08 15:01:55.251722 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.251702    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa04c39-0899-40d3-809b-5a1d03396342-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh\" (UID: \"8fa04c39-0899-40d3-809b-5a1d03396342\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"
Sep 08 15:01:55.387486 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.387412    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"
Sep 08 15:01:55.658187 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:55.658163    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"]
Sep 08 15:01:55.660260 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:01:55.660229    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa04c39_0899_40d3_809b_5a1d03396342.slice/crio-4e2f2f596e8382ca332975756ff3e58fe98b799c98c9c243b0b08da1669eedb9 WatchSource:0}: Error finding container 4e2f2f596e8382ca332975756ff3e58fe98b799c98c9c243b0b08da1669eedb9: Status 404 returned error can't find the container with id 4e2f2f596e8382ca332975756ff3e58fe98b799c98c9c243b0b08da1669eedb9
Sep 08 15:01:56.224892 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:56.224849    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh" event={"ID":"8fa04c39-0899-40d3-809b-5a1d03396342","Type":"ContainerStarted","Data":"41d76f61959bdc4b34cf90da45e3ce3dbf2810294da510abe704f52fceac90f9"}
Sep 08 15:01:56.225062 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:56.224910    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh" event={"ID":"8fa04c39-0899-40d3-809b-5a1d03396342","Type":"ContainerStarted","Data":"4e2f2f596e8382ca332975756ff3e58fe98b799c98c9c243b0b08da1669eedb9"}
Sep 08 15:01:56.241381 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:56.241297    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh" podStartSLOduration=1.241280803 podStartE2EDuration="1.241280803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:01:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:01:56.239933027 +0000 UTC m=+1666.967750065" watchObservedRunningTime="2026-09-08 15:01:56.241280803 +0000 UTC m=+1666.969097847"
Sep 08 15:01:57.232158 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:57.232124    2583 generic.go:356] "Generic (PLEG): container finished" podID="8fa04c39-0899-40d3-809b-5a1d03396342" containerID="41d76f61959bdc4b34cf90da45e3ce3dbf2810294da510abe704f52fceac90f9" exitCode=0
Sep 08 15:01:57.232533 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:57.232212    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh" event={"ID":"8fa04c39-0899-40d3-809b-5a1d03396342","Type":"ContainerDied","Data":"41d76f61959bdc4b34cf90da45e3ce3dbf2810294da510abe704f52fceac90f9"}
Sep 08 15:01:58.365521 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:58.365491    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"
Sep 08 15:01:58.481587 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:58.481555    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa04c39-0899-40d3-809b-5a1d03396342-maas-api-service-ca\") pod \"8fa04c39-0899-40d3-809b-5a1d03396342\" (UID: \"8fa04c39-0899-40d3-809b-5a1d03396342\") "
Sep 08 15:01:58.481942 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:58.481918    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fa04c39-0899-40d3-809b-5a1d03396342-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8fa04c39-0899-40d3-809b-5a1d03396342" (UID: "8fa04c39-0899-40d3-809b-5a1d03396342"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:01:58.583140 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:58.583048    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa04c39-0899-40d3-809b-5a1d03396342-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:01:59.244045 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:59.244016    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"
Sep 08 15:01:59.244045 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:59.244026    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh" event={"ID":"8fa04c39-0899-40d3-809b-5a1d03396342","Type":"ContainerDied","Data":"4e2f2f596e8382ca332975756ff3e58fe98b799c98c9c243b0b08da1669eedb9"}
Sep 08 15:01:59.244261 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:01:59.244069    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e2f2f596e8382ca332975756ff3e58fe98b799c98c9c243b0b08da1669eedb9"
Sep 08 15:02:05.634672 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:05.634636    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t"]
Sep 08 15:02:05.637626 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:05.637601    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a89cbb-29814660-2fm5t"]
Sep 08 15:02:05.669708 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:05.669676    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"]
Sep 08 15:02:05.669959 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:05.669932    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t" podUID="ce7a1447-da27-4e1a-af1d-847b6128c28f" containerName="payload-processing" containerID="cri-o://0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09" gracePeriod=30
Sep 08 15:02:05.678183 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:05.678116    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"]
Sep 08 15:02:05.678392 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:05.678366    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4" podUID="434c7944-0250-4496-8249-559af445c24e" containerName="payload-pre-processing" containerID="cri-o://2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276" gracePeriod=30
Sep 08 15:02:05.794567 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:05.794535    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05692f9b-624d-453a-b6d4-fed501485c8d" path="/var/lib/kubelet/pods/05692f9b-624d-453a-b6d4-fed501485c8d/volumes"
Sep 08 15:02:05.830200 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:05.830169    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"]
Sep 08 15:02:05.834757 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:05.834724    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a89cbb-5khnh"]
Sep 08 15:02:05.952776 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:05.952751    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 15:02:05.955572 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:05.955554    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 15:02:06.042434 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.042402    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434c7944-0250-4496-8249-559af445c24e-plugins-config-volume\") pod \"434c7944-0250-4496-8249-559af445c24e\" (UID: \"434c7944-0250-4496-8249-559af445c24e\") "
Sep 08 15:02:06.042589 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.042462    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce7a1447-da27-4e1a-af1d-847b6128c28f-plugins-config-volume\") pod \"ce7a1447-da27-4e1a-af1d-847b6128c28f\" (UID: \"ce7a1447-da27-4e1a-af1d-847b6128c28f\") "
Sep 08 15:02:06.042589 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.042541    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4bjc\" (UniqueName: \"kubernetes.io/projected/ce7a1447-da27-4e1a-af1d-847b6128c28f-kube-api-access-b4bjc\") pod \"ce7a1447-da27-4e1a-af1d-847b6128c28f\" (UID: \"ce7a1447-da27-4e1a-af1d-847b6128c28f\") "
Sep 08 15:02:06.042589 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.042572    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7msdn\" (UniqueName: \"kubernetes.io/projected/434c7944-0250-4496-8249-559af445c24e-kube-api-access-7msdn\") pod \"434c7944-0250-4496-8249-559af445c24e\" (UID: \"434c7944-0250-4496-8249-559af445c24e\") "
Sep 08 15:02:06.042897 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.042870    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/434c7944-0250-4496-8249-559af445c24e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "434c7944-0250-4496-8249-559af445c24e" (UID: "434c7944-0250-4496-8249-559af445c24e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:02:06.042897 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.042890    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce7a1447-da27-4e1a-af1d-847b6128c28f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce7a1447-da27-4e1a-af1d-847b6128c28f" (UID: "ce7a1447-da27-4e1a-af1d-847b6128c28f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:02:06.044756 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.044735    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce7a1447-da27-4e1a-af1d-847b6128c28f-kube-api-access-b4bjc" (OuterVolumeSpecName: "kube-api-access-b4bjc") pod "ce7a1447-da27-4e1a-af1d-847b6128c28f" (UID: "ce7a1447-da27-4e1a-af1d-847b6128c28f"). InnerVolumeSpecName "kube-api-access-b4bjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:02:06.044807 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.044769    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434c7944-0250-4496-8249-559af445c24e-kube-api-access-7msdn" (OuterVolumeSpecName: "kube-api-access-7msdn") pod "434c7944-0250-4496-8249-559af445c24e" (UID: "434c7944-0250-4496-8249-559af445c24e"). InnerVolumeSpecName "kube-api-access-7msdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:02:06.143976 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.143947    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4bjc\" (UniqueName: \"kubernetes.io/projected/ce7a1447-da27-4e1a-af1d-847b6128c28f-kube-api-access-b4bjc\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:02:06.143976 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.143971    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7msdn\" (UniqueName: \"kubernetes.io/projected/434c7944-0250-4496-8249-559af445c24e-kube-api-access-7msdn\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:02:06.143976 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.143983    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434c7944-0250-4496-8249-559af445c24e-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:02:06.144195 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.143991    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce7a1447-da27-4e1a-af1d-847b6128c28f-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:02:06.281342 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.281308    2583 generic.go:356] "Generic (PLEG): container finished" podID="ce7a1447-da27-4e1a-af1d-847b6128c28f" containerID="0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09" exitCode=0
Sep 08 15:02:06.281500 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.281358    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"
Sep 08 15:02:06.281500 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.281393    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t" event={"ID":"ce7a1447-da27-4e1a-af1d-847b6128c28f","Type":"ContainerDied","Data":"0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09"}
Sep 08 15:02:06.281500 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.281443    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t" event={"ID":"ce7a1447-da27-4e1a-af1d-847b6128c28f","Type":"ContainerDied","Data":"f3b13b5a2c50fdcceff57213bee0ad1e03747b71369fe4f2328a26dfd35b9ade"}
Sep 08 15:02:06.281500 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.281467    2583 scope.go:117] "RemoveContainer" containerID="0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09"
Sep 08 15:02:06.282699 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.282677    2583 generic.go:356] "Generic (PLEG): container finished" podID="434c7944-0250-4496-8249-559af445c24e" containerID="2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276" exitCode=0
Sep 08 15:02:06.282793 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.282719    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4" event={"ID":"434c7944-0250-4496-8249-559af445c24e","Type":"ContainerDied","Data":"2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276"}
Sep 08 15:02:06.282793 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.282758    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4" event={"ID":"434c7944-0250-4496-8249-559af445c24e","Type":"ContainerDied","Data":"d6618689c34a56808900eb398dc1a7a990608a5ac92d6facb1e6f7a051cf276b"}
Sep 08 15:02:06.282793 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.282720    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"
Sep 08 15:02:06.292046 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.292026    2583 scope.go:117] "RemoveContainer" containerID="0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09"
Sep 08 15:02:06.292358 ip-10-0-137-6 kubenswrapper[2583]: E0908 15:02:06.292334    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09\": container with ID starting with 0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09 not found: ID does not exist" containerID="0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09"
Sep 08 15:02:06.292437 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.292372    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09"} err="failed to get container status \"0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09\": rpc error: code = NotFound desc = could not find container \"0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09\": container with ID starting with 0646dc9a028ff756727946bd5463fbe1dd7716af311c2dc7b41a2e1cdbd00e09 not found: ID does not exist"
Sep 08 15:02:06.292437 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.292395    2583 scope.go:117] "RemoveContainer" containerID="2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276"
Sep 08 15:02:06.302397 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.302373    2583 scope.go:117] "RemoveContainer" containerID="2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276"
Sep 08 15:02:06.302680 ip-10-0-137-6 kubenswrapper[2583]: E0908 15:02:06.302662    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276\": container with ID starting with 2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276 not found: ID does not exist" containerID="2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276"
Sep 08 15:02:06.302730 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.302688    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276"} err="failed to get container status \"2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276\": rpc error: code = NotFound desc = could not find container \"2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276\": container with ID starting with 2f80f25fe6d79726f6db3f6fdf3c5cf0ff9594105c2c964a7ca0e942ee2db276 not found: ID does not exist"
Sep 08 15:02:06.308727 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.308705    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"]
Sep 08 15:02:06.314727 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.314707    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a89cbb-59b4f6b95c-q5d7t"]
Sep 08 15:02:06.327539 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.327519    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"]
Sep 08 15:02:06.332513 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:06.332491    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a89cbb-5798dd4d9d-bl4m4"]
Sep 08 15:02:07.210079 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:07.210045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:07.296693 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:07.296665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:07.793154 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:07.793123    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="434c7944-0250-4496-8249-559af445c24e" path="/var/lib/kubelet/pods/434c7944-0250-4496-8249-559af445c24e/volumes"
Sep 08 15:02:07.793447 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:07.793434    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fa04c39-0899-40d3-809b-5a1d03396342" path="/var/lib/kubelet/pods/8fa04c39-0899-40d3-809b-5a1d03396342/volumes"
Sep 08 15:02:07.793743 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:07.793732    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce7a1447-da27-4e1a-af1d-847b6128c28f" path="/var/lib/kubelet/pods/ce7a1447-da27-4e1a-af1d-847b6128c28f/volumes"
Sep 08 15:02:12.259107 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:12.259071    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:12.364722 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:12.364692    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:12.554476 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:12.554391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:13.062707 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:13.062671    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:23.549227 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:23.549182    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:23.581174 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:23.581141    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"]
Sep 08 15:02:23.581405 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:23.581367    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl" podUID="638d0163-dc0b-45ea-8566-7b5ced202652" containerName="payload-pre-processing" containerID="cri-o://88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8" gracePeriod=30
Sep 08 15:02:23.650215 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:23.650183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:23.770117 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:23.770086    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:23.829189 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:23.829167    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 15:02:23.855282 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:23.855253    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:24.001523 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.001488    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s78sc\" (UniqueName: \"kubernetes.io/projected/638d0163-dc0b-45ea-8566-7b5ced202652-kube-api-access-s78sc\") pod \"638d0163-dc0b-45ea-8566-7b5ced202652\" (UID: \"638d0163-dc0b-45ea-8566-7b5ced202652\") "
Sep 08 15:02:24.001709 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.001592    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/638d0163-dc0b-45ea-8566-7b5ced202652-plugins-config-volume\") pod \"638d0163-dc0b-45ea-8566-7b5ced202652\" (UID: \"638d0163-dc0b-45ea-8566-7b5ced202652\") "
Sep 08 15:02:24.002018 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.001997    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/638d0163-dc0b-45ea-8566-7b5ced202652-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "638d0163-dc0b-45ea-8566-7b5ced202652" (UID: "638d0163-dc0b-45ea-8566-7b5ced202652"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:02:24.003780 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.003756    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/638d0163-dc0b-45ea-8566-7b5ced202652-kube-api-access-s78sc" (OuterVolumeSpecName: "kube-api-access-s78sc") pod "638d0163-dc0b-45ea-8566-7b5ced202652" (UID: "638d0163-dc0b-45ea-8566-7b5ced202652"). InnerVolumeSpecName "kube-api-access-s78sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:02:24.102673 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.102596    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s78sc\" (UniqueName: \"kubernetes.io/projected/638d0163-dc0b-45ea-8566-7b5ced202652-kube-api-access-s78sc\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:02:24.102673 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.102623    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/638d0163-dc0b-45ea-8566-7b5ced202652-plugins-config-volume\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:02:24.177826 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.177791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:24.383452 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.383365    2583 generic.go:356] "Generic (PLEG): container finished" podID="638d0163-dc0b-45ea-8566-7b5ced202652" containerID="88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8" exitCode=0
Sep 08 15:02:24.383452 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.383410    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl" event={"ID":"638d0163-dc0b-45ea-8566-7b5ced202652","Type":"ContainerDied","Data":"88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8"}
Sep 08 15:02:24.383626 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.383453    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl" event={"ID":"638d0163-dc0b-45ea-8566-7b5ced202652","Type":"ContainerDied","Data":"1a0528001603e208c378a79f88b59c4e383e9c34a3bd4741e19e883c83f61662"}
Sep 08 15:02:24.383626 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.383474    2583 scope.go:117] "RemoveContainer" containerID="88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8"
Sep 08 15:02:24.383626 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.383413    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"
Sep 08 15:02:24.392983 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.392964    2583 scope.go:117] "RemoveContainer" containerID="88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8"
Sep 08 15:02:24.393263 ip-10-0-137-6 kubenswrapper[2583]: E0908 15:02:24.393244    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8\": container with ID starting with 88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8 not found: ID does not exist" containerID="88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8"
Sep 08 15:02:24.393310 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.393272    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8"} err="failed to get container status \"88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8\": rpc error: code = NotFound desc = could not find container \"88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8\": container with ID starting with 88f809c2543cc1eb09af8ebee41aa9cfcb9349c6b05fd7f3e05bb4772e7215c8 not found: ID does not exist"
Sep 08 15:02:24.406106 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.406079    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"]
Sep 08 15:02:24.410428 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:24.410406    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ef8f61-7cfdb77f78-vq5jl"]
Sep 08 15:02:25.793325 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:25.793291    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="638d0163-dc0b-45ea-8566-7b5ced202652" path="/var/lib/kubelet/pods/638d0163-dc0b-45ea-8566-7b5ced202652/volumes"
Sep 08 15:02:31.581996 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:31.581915    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:31.741895 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:31.741849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:31.917479 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:31.917398    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:32.267731 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.267696    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"]
Sep 08 15:02:32.268073 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268060    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fa04c39-0899-40d3-809b-5a1d03396342" containerName="revoke-keys"
Sep 08 15:02:32.268129 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268075    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa04c39-0899-40d3-809b-5a1d03396342" containerName="revoke-keys"
Sep 08 15:02:32.268129 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268087    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="638d0163-dc0b-45ea-8566-7b5ced202652" containerName="payload-pre-processing"
Sep 08 15:02:32.268129 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268092    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="638d0163-dc0b-45ea-8566-7b5ced202652" containerName="payload-pre-processing"
Sep 08 15:02:32.268129 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268099    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce7a1447-da27-4e1a-af1d-847b6128c28f" containerName="payload-processing"
Sep 08 15:02:32.268129 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268105    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce7a1447-da27-4e1a-af1d-847b6128c28f" containerName="payload-processing"
Sep 08 15:02:32.268129 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268119    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="434c7944-0250-4496-8249-559af445c24e" containerName="payload-pre-processing"
Sep 08 15:02:32.268129 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268124    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="434c7944-0250-4496-8249-559af445c24e" containerName="payload-pre-processing"
Sep 08 15:02:32.268341 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268181    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce7a1447-da27-4e1a-af1d-847b6128c28f" containerName="payload-processing"
Sep 08 15:02:32.268341 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268189    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="638d0163-dc0b-45ea-8566-7b5ced202652" containerName="payload-pre-processing"
Sep 08 15:02:32.268341 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268196    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fa04c39-0899-40d3-809b-5a1d03396342" containerName="revoke-keys"
Sep 08 15:02:32.268341 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.268205    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="434c7944-0250-4496-8249-559af445c24e" containerName="payload-pre-processing"
Sep 08 15:02:32.272422 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.272402    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"
Sep 08 15:02:32.275664 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.275642    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:02:32.277388 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.277371    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pdpbl\""
Sep 08 15:02:32.287741 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.287718    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"]
Sep 08 15:02:32.373596 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.373565    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d74ab3de-02d1-4901-9a31-8e28f99b856d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26\" (UID: \"d74ab3de-02d1-4901-9a31-8e28f99b856d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"
Sep 08 15:02:32.426500 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.426470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:32.474749 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.474726    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d74ab3de-02d1-4901-9a31-8e28f99b856d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26\" (UID: \"d74ab3de-02d1-4901-9a31-8e28f99b856d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"
Sep 08 15:02:32.475287 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.475269    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d74ab3de-02d1-4901-9a31-8e28f99b856d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26\" (UID: \"d74ab3de-02d1-4901-9a31-8e28f99b856d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"
Sep 08 15:02:32.582593 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.582506    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"
Sep 08 15:02:32.887708 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:32.887633    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"]
Sep 08 15:02:32.899959 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:02:32.899927    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd74ab3de_02d1_4901_9a31_8e28f99b856d.slice/crio-ca330285f8ae752dc7e09fee35e72bf315bac700679aaa7d3d94923593b9563e WatchSource:0}: Error finding container ca330285f8ae752dc7e09fee35e72bf315bac700679aaa7d3d94923593b9563e: Status 404 returned error can't find the container with id ca330285f8ae752dc7e09fee35e72bf315bac700679aaa7d3d94923593b9563e
Sep 08 15:02:33.432237 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:33.432205    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26" event={"ID":"d74ab3de-02d1-4901-9a31-8e28f99b856d","Type":"ContainerStarted","Data":"4601d81d6c53cd8bd9a6a7916eb133d258fccb08bc92af653c99d4512dcfa9c3"}
Sep 08 15:02:33.432414 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:33.432261    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26" event={"ID":"d74ab3de-02d1-4901-9a31-8e28f99b856d","Type":"ContainerStarted","Data":"ca330285f8ae752dc7e09fee35e72bf315bac700679aaa7d3d94923593b9563e"}
Sep 08 15:02:33.467172 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:33.467120    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26" podStartSLOduration=1.467105935 podStartE2EDuration="1.467105935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:02:33.466643522 +0000 UTC m=+1704.194460537" watchObservedRunningTime="2026-09-08 15:02:33.467105935 +0000 UTC m=+1704.194922950"
Sep 08 15:02:34.438495 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:34.438460    2583 generic.go:356] "Generic (PLEG): container finished" podID="d74ab3de-02d1-4901-9a31-8e28f99b856d" containerID="4601d81d6c53cd8bd9a6a7916eb133d258fccb08bc92af653c99d4512dcfa9c3" exitCode=0
Sep 08 15:02:34.438888 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:34.438545    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26" event={"ID":"d74ab3de-02d1-4901-9a31-8e28f99b856d","Type":"ContainerDied","Data":"4601d81d6c53cd8bd9a6a7916eb133d258fccb08bc92af653c99d4512dcfa9c3"}
Sep 08 15:02:35.575494 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:35.575473    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"
Sep 08 15:02:35.599749 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:35.599725    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d74ab3de-02d1-4901-9a31-8e28f99b856d-maas-api-service-ca\") pod \"d74ab3de-02d1-4901-9a31-8e28f99b856d\" (UID: \"d74ab3de-02d1-4901-9a31-8e28f99b856d\") "
Sep 08 15:02:35.600072 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:35.600050    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d74ab3de-02d1-4901-9a31-8e28f99b856d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d74ab3de-02d1-4901-9a31-8e28f99b856d" (UID: "d74ab3de-02d1-4901-9a31-8e28f99b856d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:02:35.701007 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:35.700936    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d74ab3de-02d1-4901-9a31-8e28f99b856d-maas-api-service-ca\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:02:36.449785 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:36.449753    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"
Sep 08 15:02:36.450051 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:36.449750    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26" event={"ID":"d74ab3de-02d1-4901-9a31-8e28f99b856d","Type":"ContainerDied","Data":"ca330285f8ae752dc7e09fee35e72bf315bac700679aaa7d3d94923593b9563e"}
Sep 08 15:02:36.450051 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:36.449886    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca330285f8ae752dc7e09fee35e72bf315bac700679aaa7d3d94923593b9563e"
Sep 08 15:02:42.768299 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:42.768260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:42.848244 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:42.848208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:42.920441 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:42.920402    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:42.960974 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:42.960948    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"]
Sep 08 15:02:42.963991 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:42.963968    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f83533-h9v26"]
Sep 08 15:02:42.997850 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:42.997807    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:43.369527 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:43.369490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:43.793546 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:43.793511    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d74ab3de-02d1-4901-9a31-8e28f99b856d" path="/var/lib/kubelet/pods/d74ab3de-02d1-4901-9a31-8e28f99b856d/volumes"
Sep 08 15:02:49.515362 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:49.515325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:49.707895 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:49.707860    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:02:52.658806 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:02:52.658776    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:03:10.078602 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:03:10.078562    2583 scope.go:117] "RemoveContainer" containerID="5c725d431cedda314d198e944d28d91f61eb20c3ec68967066ccfab1e9dbf217"
Sep 08 15:03:10.087213 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:03:10.087192    2583 scope.go:117] "RemoveContainer" containerID="d9348ffc26f6b1e97825a2a00c8957ca4065b0f88b4b9dac6e1d4caec59749a4"
Sep 08 15:04:10.114807 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:04:10.114776    2583 scope.go:117] "RemoveContainer" containerID="a4239a7e2d2bb5e2f8f1a7e4d2172df66385d5331f57547fbc52cbcf8ce1b398"
Sep 08 15:04:10.123551 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:04:10.123528    2583 scope.go:117] "RemoveContainer" containerID="328ef3ea92ae12b12e690bbedb08b9f5fdac8d6571cb7186bcedec9533bd07a0"
Sep 08 15:04:13.192427 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:04:13.192390    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:04:13.878275 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:04:13.878240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:04:15.520616 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:04:15.520585    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:05:10.654203 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:05:10.654159    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:05:18.560756 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:05:18.560716    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:05:48.085533 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:05:48.085498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:05:48.196114 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:05:48.196029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:06:10.153881 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:06:10.153849    2583 scope.go:117] "RemoveContainer" containerID="61ee4e20d87b61d8f3f5c936bbeb070c1cd15a3807a031102058fcb10e820d2e"
Sep 08 15:06:10.162490 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:06:10.162468    2583 scope.go:117] "RemoveContainer" containerID="1a2f6a1234d760c83def8bd2db20a8f5e90da7326f1db77270a62ce6ad6bf7da"
Sep 08 15:06:24.105499 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:06:24.105466    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:06:24.157376 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:06:24.157346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:07:10.186233 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:10.186201    2583 scope.go:117] "RemoveContainer" containerID="332ff815c8ca4a2319e1e9c2d565261793b30340d524fa9d10f2d3e3e8a9f260"
Sep 08 15:07:10.194580 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:10.194560    2583 scope.go:117] "RemoveContainer" containerID="f1ae5ac4ee705e9164d89bc263b70e0ee4eebc3351bd4ee591242d00187942af"
Sep 08 15:07:10.202681 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:10.202660    2583 scope.go:117] "RemoveContainer" containerID="441aecb4a3b14e9e1ea7ad898d1aef2d46ba17d2ccac744c871f99493944f9e2"
Sep 08 15:07:16.856666 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.856632    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"]
Sep 08 15:07:16.857057 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.857042    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d74ab3de-02d1-4901-9a31-8e28f99b856d" containerName="revoke-keys"
Sep 08 15:07:16.857099 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.857059    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d74ab3de-02d1-4901-9a31-8e28f99b856d" containerName="revoke-keys"
Sep 08 15:07:16.857169 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.857155    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d74ab3de-02d1-4901-9a31-8e28f99b856d" containerName="revoke-keys"
Sep 08 15:07:16.859977 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.859957    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:16.862896 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.862874    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-4f328d8e-kserve-self-signed-certs\""
Sep 08 15:07:16.872629 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.872608    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"]
Sep 08 15:07:16.893353 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.893322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:07:16.911149 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.911119    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11a4810e-6ab5-4991-b426-38dde4220c68-tls-certs\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:16.911242 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.911154    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glln6\" (UniqueName: \"kubernetes.io/projected/11a4810e-6ab5-4991-b426-38dde4220c68-kube-api-access-glln6\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:16.911242 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.911177    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-dshm\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:16.911316 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.911274    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-model-cache\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:16.911356 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.911311    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-home\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:16.911420 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:16.911402    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-tmp-dir\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.012203 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.012176    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-model-cache\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.012203 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.012208    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-home\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.012408 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.012246    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-tmp-dir\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.012408 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.012275    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11a4810e-6ab5-4991-b426-38dde4220c68-tls-certs\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.012408 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.012296    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glln6\" (UniqueName: \"kubernetes.io/projected/11a4810e-6ab5-4991-b426-38dde4220c68-kube-api-access-glln6\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.012408 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.012319    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-dshm\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.012642 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.012619    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-model-cache\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.012703 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.012641    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-home\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.012771 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.012716    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-tmp-dir\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.014644 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.014624    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-dshm\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.014752 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.014735    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11a4810e-6ab5-4991-b426-38dde4220c68-tls-certs\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.022012 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.021984    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glln6\" (UniqueName: \"kubernetes.io/projected/11a4810e-6ab5-4991-b426-38dde4220c68-kube-api-access-glln6\") pod \"e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") " pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.170823 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.170728    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:17.442649 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.442616    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"]
Sep 08 15:07:17.447105 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:07:17.447079    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11a4810e_6ab5_4991_b426_38dde4220c68.slice/crio-c68c00d4694ea7a2aaad0709e50181226dd4f055ef7d44839328e66438ccd8c8 WatchSource:0}: Error finding container c68c00d4694ea7a2aaad0709e50181226dd4f055ef7d44839328e66438ccd8c8: Status 404 returned error can't find the container with id c68c00d4694ea7a2aaad0709e50181226dd4f055ef7d44839328e66438ccd8c8
Sep 08 15:07:17.871632 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.871595    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2" event={"ID":"11a4810e-6ab5-4991-b426-38dde4220c68","Type":"ContainerStarted","Data":"17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195"}
Sep 08 15:07:17.871632 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.871638    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2" event={"ID":"11a4810e-6ab5-4991-b426-38dde4220c68","Type":"ContainerStarted","Data":"c68c00d4694ea7a2aaad0709e50181226dd4f055ef7d44839328e66438ccd8c8"}
Sep 08 15:07:17.891350 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:17.891302    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2" podStartSLOduration=1.8912881609999999 podStartE2EDuration="1.891288161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:07:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:07:17.889236291 +0000 UTC m=+1988.617053305" watchObservedRunningTime="2026-09-08 15:07:17.891288161 +0000 UTC m=+1988.619105175"
Sep 08 15:07:19.585349 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:19.585304    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:07:27.171044 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:27.171002    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:27.171414 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:27.171060    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:27.183543 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:27.183520    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:27.930246 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:27.930214    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:07:29.561016 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.560979    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"]
Sep 08 15:07:29.565144 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.565128    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.568103 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.568083    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-4f328d8e-kserve-self-signed-certs\""
Sep 08 15:07:29.574445 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.574421    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"]
Sep 08 15:07:29.588703 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.588662    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:07:29.636806 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.636778    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-dshm\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.636958 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.636822    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhszs\" (UniqueName: \"kubernetes.io/projected/4486d43b-4900-4152-8706-0cb4ba8895cd-kube-api-access-vhszs\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.636958 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.636869    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-model-cache\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.636958 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.636946    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-home\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.637131 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.636984    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4486d43b-4900-4152-8706-0cb4ba8895cd-tls-certs\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.637131 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.637012    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-tmp-dir\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.737680 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.737649    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-dshm\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.737804 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.737688    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhszs\" (UniqueName: \"kubernetes.io/projected/4486d43b-4900-4152-8706-0cb4ba8895cd-kube-api-access-vhszs\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.737804 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.737711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-model-cache\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.737804 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.737746    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-home\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.737804 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.737774    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4486d43b-4900-4152-8706-0cb4ba8895cd-tls-certs\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.738051 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.737806    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-tmp-dir\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.738167 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.738143    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-model-cache\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.738234 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.738220    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-home\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.738273 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.738238    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-tmp-dir\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.740024 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.740004    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-dshm\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.740359 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.740337    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4486d43b-4900-4152-8706-0cb4ba8895cd-tls-certs\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.745861 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.745824    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhszs\" (UniqueName: \"kubernetes.io/projected/4486d43b-4900-4152-8706-0cb4ba8895cd-kube-api-access-vhszs\") pod \"e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") " pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:29.875690 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:29.875610    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:30.148537 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:30.148483    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"]
Sep 08 15:07:30.150637 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:07:30.150605    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4486d43b_4900_4152_8706_0cb4ba8895cd.slice/crio-4118adaa4d5631302047a9105476cd0c32b56654050c33ec84b592e7b36b68dd WatchSource:0}: Error finding container 4118adaa4d5631302047a9105476cd0c32b56654050c33ec84b592e7b36b68dd: Status 404 returned error can't find the container with id 4118adaa4d5631302047a9105476cd0c32b56654050c33ec84b592e7b36b68dd
Sep 08 15:07:30.936526 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:30.936486    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx" event={"ID":"4486d43b-4900-4152-8706-0cb4ba8895cd","Type":"ContainerStarted","Data":"4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d"}
Sep 08 15:07:30.936947 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:30.936542    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx" event={"ID":"4486d43b-4900-4152-8706-0cb4ba8895cd","Type":"ContainerStarted","Data":"4118adaa4d5631302047a9105476cd0c32b56654050c33ec84b592e7b36b68dd"}
Sep 08 15:07:30.963114 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:30.963068    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx" podStartSLOduration=1.963053997 podStartE2EDuration="1.963053997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:07:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:07:30.962143091 +0000 UTC m=+2001.689960106" watchObservedRunningTime="2026-09-08 15:07:30.963053997 +0000 UTC m=+2001.690871011"
Sep 08 15:07:39.876197 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:39.876163    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:39.876692 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:39.876239    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:39.888788 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:39.888765    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:39.994794 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:39.994766    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:07:53.207370 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:53.207331    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"]
Sep 08 15:07:53.228631 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:53.228605    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:07:53.846057 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:53.846018    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"]
Sep 08 15:07:53.862300 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:07:53.862265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:08:08.228018 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.227958    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2" podUID="11a4810e-6ab5-4991-b426-38dde4220c68" containerName="main" containerID="cri-o://17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195" gracePeriod=45
Sep 08 15:08:08.474416 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.474392    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:08:08.573750 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.573678    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-dshm\") pod \"11a4810e-6ab5-4991-b426-38dde4220c68\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") "
Sep 08 15:08:08.573750 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.573719    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-home\") pod \"11a4810e-6ab5-4991-b426-38dde4220c68\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") "
Sep 08 15:08:08.574004 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.573763    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glln6\" (UniqueName: \"kubernetes.io/projected/11a4810e-6ab5-4991-b426-38dde4220c68-kube-api-access-glln6\") pod \"11a4810e-6ab5-4991-b426-38dde4220c68\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") "
Sep 08 15:08:08.574004 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.573856    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-tmp-dir\") pod \"11a4810e-6ab5-4991-b426-38dde4220c68\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") "
Sep 08 15:08:08.574004 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.573887    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11a4810e-6ab5-4991-b426-38dde4220c68-tls-certs\") pod \"11a4810e-6ab5-4991-b426-38dde4220c68\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") "
Sep 08 15:08:08.574004 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.573934    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-model-cache\") pod \"11a4810e-6ab5-4991-b426-38dde4220c68\" (UID: \"11a4810e-6ab5-4991-b426-38dde4220c68\") "
Sep 08 15:08:08.574205 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.574061    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-home" (OuterVolumeSpecName: "home") pod "11a4810e-6ab5-4991-b426-38dde4220c68" (UID: "11a4810e-6ab5-4991-b426-38dde4220c68"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:08:08.574276 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.574215    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "11a4810e-6ab5-4991-b426-38dde4220c68" (UID: "11a4810e-6ab5-4991-b426-38dde4220c68"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:08:08.574276 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.574230    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-model-cache" (OuterVolumeSpecName: "model-cache") pod "11a4810e-6ab5-4991-b426-38dde4220c68" (UID: "11a4810e-6ab5-4991-b426-38dde4220c68"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:08:08.574276 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.574271    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:08.575918 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.575893    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-dshm" (OuterVolumeSpecName: "dshm") pod "11a4810e-6ab5-4991-b426-38dde4220c68" (UID: "11a4810e-6ab5-4991-b426-38dde4220c68"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:08:08.576107 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.576088    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11a4810e-6ab5-4991-b426-38dde4220c68-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "11a4810e-6ab5-4991-b426-38dde4220c68" (UID: "11a4810e-6ab5-4991-b426-38dde4220c68"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:08:08.576162 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.576144    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a4810e-6ab5-4991-b426-38dde4220c68-kube-api-access-glln6" (OuterVolumeSpecName: "kube-api-access-glln6") pod "11a4810e-6ab5-4991-b426-38dde4220c68" (UID: "11a4810e-6ab5-4991-b426-38dde4220c68"). InnerVolumeSpecName "kube-api-access-glln6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:08:08.675540 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.675508    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:08.675540 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.675535    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glln6\" (UniqueName: \"kubernetes.io/projected/11a4810e-6ab5-4991-b426-38dde4220c68-kube-api-access-glln6\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:08.675540 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.675546    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:08.675744 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.675554    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11a4810e-6ab5-4991-b426-38dde4220c68-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:08.675744 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.675562    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11a4810e-6ab5-4991-b426-38dde4220c68-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:08.866277 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:08.866166    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx" podUID="4486d43b-4900-4152-8706-0cb4ba8895cd" containerName="main" containerID="cri-o://4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d" gracePeriod=45
Sep 08 15:08:09.103950 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.103926    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:08:09.125418 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.125357    2583 generic.go:356] "Generic (PLEG): container finished" podID="11a4810e-6ab5-4991-b426-38dde4220c68" containerID="17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195" exitCode=0
Sep 08 15:08:09.125418 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.125397    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"
Sep 08 15:08:09.125579 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.125436    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2" event={"ID":"11a4810e-6ab5-4991-b426-38dde4220c68","Type":"ContainerDied","Data":"17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195"}
Sep 08 15:08:09.125579 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.125470    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2" event={"ID":"11a4810e-6ab5-4991-b426-38dde4220c68","Type":"ContainerDied","Data":"c68c00d4694ea7a2aaad0709e50181226dd4f055ef7d44839328e66438ccd8c8"}
Sep 08 15:08:09.125579 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.125489    2583 scope.go:117] "RemoveContainer" containerID="17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195"
Sep 08 15:08:09.126769 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.126740    2583 generic.go:356] "Generic (PLEG): container finished" podID="4486d43b-4900-4152-8706-0cb4ba8895cd" containerID="4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d" exitCode=0
Sep 08 15:08:09.126919 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.126769    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx" event={"ID":"4486d43b-4900-4152-8706-0cb4ba8895cd","Type":"ContainerDied","Data":"4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d"}
Sep 08 15:08:09.126919 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.126787    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"
Sep 08 15:08:09.126919 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.126810    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx" event={"ID":"4486d43b-4900-4152-8706-0cb4ba8895cd","Type":"ContainerDied","Data":"4118adaa4d5631302047a9105476cd0c32b56654050c33ec84b592e7b36b68dd"}
Sep 08 15:08:09.134314 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.134296    2583 scope.go:117] "RemoveContainer" containerID="17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195"
Sep 08 15:08:09.134591 ip-10-0-137-6 kubenswrapper[2583]: E0908 15:08:09.134573    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195\": container with ID starting with 17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195 not found: ID does not exist" containerID="17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195"
Sep 08 15:08:09.134663 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.134598    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195"} err="failed to get container status \"17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195\": rpc error: code = NotFound desc = could not find container \"17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195\": container with ID starting with 17edeaac2392d735fbd4bcac7fb361a99a0b5b426cbfcf097c29d5064395e195 not found: ID does not exist"
Sep 08 15:08:09.134663 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.134616    2583 scope.go:117] "RemoveContainer" containerID="4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d"
Sep 08 15:08:09.143824 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.143804    2583 scope.go:117] "RemoveContainer" containerID="4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d"
Sep 08 15:08:09.144154 ip-10-0-137-6 kubenswrapper[2583]: E0908 15:08:09.144133    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d\": container with ID starting with 4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d not found: ID does not exist" containerID="4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d"
Sep 08 15:08:09.144295 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.144159    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d"} err="failed to get container status \"4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d\": rpc error: code = NotFound desc = could not find container \"4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d\": container with ID starting with 4d226ebd8f57961c5a7f4137fb0bbd4cfc95facf68094b4e7770115c68e1596d not found: ID does not exist"
Sep 08 15:08:09.162477 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.162450    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"]
Sep 08 15:08:09.166515 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.166494    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-4f328d8e-kserve-79d8d84bdc-fs4h2"]
Sep 08 15:08:09.180273 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.180256    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4486d43b-4900-4152-8706-0cb4ba8895cd-tls-certs\") pod \"4486d43b-4900-4152-8706-0cb4ba8895cd\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") "
Sep 08 15:08:09.180374 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.180325    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-tmp-dir\") pod \"4486d43b-4900-4152-8706-0cb4ba8895cd\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") "
Sep 08 15:08:09.180374 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.180351    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-home\") pod \"4486d43b-4900-4152-8706-0cb4ba8895cd\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") "
Sep 08 15:08:09.180471 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.180379    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhszs\" (UniqueName: \"kubernetes.io/projected/4486d43b-4900-4152-8706-0cb4ba8895cd-kube-api-access-vhszs\") pod \"4486d43b-4900-4152-8706-0cb4ba8895cd\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") "
Sep 08 15:08:09.180471 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.180397    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-dshm\") pod \"4486d43b-4900-4152-8706-0cb4ba8895cd\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") "
Sep 08 15:08:09.180471 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.180430    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-model-cache\") pod \"4486d43b-4900-4152-8706-0cb4ba8895cd\" (UID: \"4486d43b-4900-4152-8706-0cb4ba8895cd\") "
Sep 08 15:08:09.180667 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.180642    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-home" (OuterVolumeSpecName: "home") pod "4486d43b-4900-4152-8706-0cb4ba8895cd" (UID: "4486d43b-4900-4152-8706-0cb4ba8895cd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:08:09.180667 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.180655    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4486d43b-4900-4152-8706-0cb4ba8895cd" (UID: "4486d43b-4900-4152-8706-0cb4ba8895cd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:08:09.180887 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.180857    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-model-cache" (OuterVolumeSpecName: "model-cache") pod "4486d43b-4900-4152-8706-0cb4ba8895cd" (UID: "4486d43b-4900-4152-8706-0cb4ba8895cd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:08:09.182577 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.182551    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-dshm" (OuterVolumeSpecName: "dshm") pod "4486d43b-4900-4152-8706-0cb4ba8895cd" (UID: "4486d43b-4900-4152-8706-0cb4ba8895cd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:08:09.182655 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.182606    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4486d43b-4900-4152-8706-0cb4ba8895cd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4486d43b-4900-4152-8706-0cb4ba8895cd" (UID: "4486d43b-4900-4152-8706-0cb4ba8895cd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:08:09.182655 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.182623    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4486d43b-4900-4152-8706-0cb4ba8895cd-kube-api-access-vhszs" (OuterVolumeSpecName: "kube-api-access-vhszs") pod "4486d43b-4900-4152-8706-0cb4ba8895cd" (UID: "4486d43b-4900-4152-8706-0cb4ba8895cd"). InnerVolumeSpecName "kube-api-access-vhszs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:08:09.282292 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.282257    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-tmp-dir\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:09.282292 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.282292    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-home\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:09.282647 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.282307    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhszs\" (UniqueName: \"kubernetes.io/projected/4486d43b-4900-4152-8706-0cb4ba8895cd-kube-api-access-vhszs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:09.282647 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.282323    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-dshm\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:09.282647 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.282336    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4486d43b-4900-4152-8706-0cb4ba8895cd-model-cache\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:09.282647 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.282348    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4486d43b-4900-4152-8706-0cb4ba8895cd-tls-certs\") on node \"ip-10-0-137-6.ec2.internal\" DevicePath \"\""
Sep 08 15:08:09.453337 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.453304    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"]
Sep 08 15:08:09.457380 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.457356    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-4f328d8e-kserve-cb4bb9f95-scxzx"]
Sep 08 15:08:09.794669 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.794636    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11a4810e-6ab5-4991-b426-38dde4220c68" path="/var/lib/kubelet/pods/11a4810e-6ab5-4991-b426-38dde4220c68/volumes"
Sep 08 15:08:09.795088 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:09.795073    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4486d43b-4900-4152-8706-0cb4ba8895cd" path="/var/lib/kubelet/pods/4486d43b-4900-4152-8706-0cb4ba8895cd/volumes"
Sep 08 15:08:10.227482 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:10.227452    2583 scope.go:117] "RemoveContainer" containerID="41d76f61959bdc4b34cf90da45e3ce3dbf2810294da510abe704f52fceac90f9"
Sep 08 15:08:43.191490 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:43.191402    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f4b698f7b-lz9v7_f47c8902-1414-4630-811b-b684dc5596fa/authorino/0.log"
Sep 08 15:08:48.732024 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:48.731993    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9pf4x_c91d268d-4b1a-4d68-ad3e-fc89ba9f4318/server/0.log"
Sep 08 15:08:49.211150 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:49.211071    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-766f77f8db-fdkgr_ae5ee9f8-55b4-4916-8cc4-a705216385e5/copy-manifests/0.log"
Sep 08 15:08:49.219208 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:49.219183    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-766f77f8db-fdkgr_ae5ee9f8-55b4-4916-8cc4-a705216385e5/manager/0.log"
Sep 08 15:08:49.869032 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:49.868999    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29814660-5m5pq_e0432e3a-a613-4fe1-b2c4-3bc17598281f/cleanup/0.log"
Sep 08 15:08:50.737607 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:50.737579    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f4b698f7b-lz9v7_f47c8902-1414-4630-811b-b684dc5596fa/authorino/0.log"
Sep 08 15:08:51.983380 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:51.983346    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dd8bb6964-pskvx_91ead5be-c3b9-4f61-931f-75199eeeae6f/kube-auth-proxy/0.log"
Sep 08 15:08:52.334330 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:52.334253    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-558h6_0ed5538c-88ce-4137-860a-1162e325b412/payload-pre-processing/0.log"
Sep 08 15:08:52.566825 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:52.566794    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65fd4f6866-nr2wr_fea5d1a3-561b-40e9-9978-9f1e662ef0c9/router/0.log"
Sep 08 15:08:53.006242 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:53.006211    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-l47fw_f6ad2fef-2603-4a1b-ad6d-07d2e9c9f092/main/0.log"
Sep 08 15:08:53.328424 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:53.328349    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs9xxx_d2365d49-879c-46cd-b363-02d80acf695c/main/0.log"
Sep 08 15:08:53.443647 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:53.443619    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-r4pwc_1ae8799f-6358-4fbb-a733-3123a72f1807/main/0.log"
Sep 08 15:08:53.556441 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:53.556413    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-748tl_44388934-a2dd-427d-9cb6-ca9d6d0e637d/main/0.log"
Sep 08 15:08:59.013335 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:59.013297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:08:59.112637 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:08:59.112601    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:04.176014 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:04.175982    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ntzrt_22348c24-e76f-4cd1-8646-7f3338b45ec7/global-pull-secret-syncer/0.log"
Sep 08 15:09:04.317164 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:04.317138    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log"
Sep 08 15:09:04.438272 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:04.438196    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-6.ec2.internal_358c2d635fdf18050ee800796677d30a/haproxy/0.log"
Sep 08 15:09:09.057048 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:09.057011    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f4b698f7b-lz9v7_f47c8902-1414-4630-811b-b684dc5596fa/authorino/0.log"
Sep 08 15:09:10.253799 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.253763    2583 scope.go:117] "RemoveContainer" containerID="4601d81d6c53cd8bd9a6a7916eb133d258fccb08bc92af653c99d4512dcfa9c3"
Sep 08 15:09:10.652153 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.652081    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d2f29701-c5b7-465a-9b76-8143abd61e92/alertmanager/0.log"
Sep 08 15:09:10.674230 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.674208    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d2f29701-c5b7-465a-9b76-8143abd61e92/config-reloader/0.log"
Sep 08 15:09:10.694688 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.694663    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d2f29701-c5b7-465a-9b76-8143abd61e92/kube-rbac-proxy-web/0.log"
Sep 08 15:09:10.722022 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.722000    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d2f29701-c5b7-465a-9b76-8143abd61e92/kube-rbac-proxy/0.log"
Sep 08 15:09:10.744980 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.744962    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d2f29701-c5b7-465a-9b76-8143abd61e92/kube-rbac-proxy-metric/0.log"
Sep 08 15:09:10.767236 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.767212    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d2f29701-c5b7-465a-9b76-8143abd61e92/prom-label-proxy/0.log"
Sep 08 15:09:10.788709 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.788692    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d2f29701-c5b7-465a-9b76-8143abd61e92/init-config-reloader/0.log"
Sep 08 15:09:10.839405 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.839382    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-lzg2x_f371cfd8-3a5b-45d1-ad8b-04d454e09460/cluster-monitoring-operator/0.log"
Sep 08 15:09:10.867759 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.867735    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hbf82_b7148135-b929-4dfe-b3ff-93a3205450cf/kube-state-metrics/0.log"
Sep 08 15:09:10.891187 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.891164    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hbf82_b7148135-b929-4dfe-b3ff-93a3205450cf/kube-rbac-proxy-main/0.log"
Sep 08 15:09:10.921725 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:10.921672    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hbf82_b7148135-b929-4dfe-b3ff-93a3205450cf/kube-rbac-proxy-self/0.log"
Sep 08 15:09:11.054971 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:11.054946    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8dppw_6e407e22-2236-4c98-abd0-d4645eec70c9/node-exporter/0.log"
Sep 08 15:09:11.094298 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:11.094278    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8dppw_6e407e22-2236-4c98-abd0-d4645eec70c9/kube-rbac-proxy/0.log"
Sep 08 15:09:11.130428 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:11.130405    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8dppw_6e407e22-2236-4c98-abd0-d4645eec70c9/init-textfile/0.log"
Sep 08 15:09:11.307302 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:11.307273    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-fdmj2_fb7e4ffc-abba-4f14-b182-bd9d4987f430/kube-rbac-proxy-main/0.log"
Sep 08 15:09:11.327347 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:11.327321    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-fdmj2_fb7e4ffc-abba-4f14-b182-bd9d4987f430/kube-rbac-proxy-self/0.log"
Sep 08 15:09:11.355276 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:11.355256    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-fdmj2_fb7e4ffc-abba-4f14-b182-bd9d4987f430/openshift-state-metrics/0.log"
Sep 08 15:09:11.682103 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:11.682017    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-qwhhf_ace0cfa7-8c9f-4c76-be70-0488de3d5401/prometheus-operator-admission-webhook/0.log"
Sep 08 15:09:13.191142 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:13.191115    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-mj4rs_3094d0b1-9ac4-42d5-824a-0fe574f90432/networking-console-plugin/0.log"
Sep 08 15:09:14.340998 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:14.340969    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-fgvm2_d918bc86-9cab-4a74-85fd-6482f08cb282/download-server/0.log"
Sep 08 15:09:16.197278 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.197243    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"]
Sep 08 15:09:16.197678 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.197651    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4486d43b-4900-4152-8706-0cb4ba8895cd" containerName="main"
Sep 08 15:09:16.197678 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.197667    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4486d43b-4900-4152-8706-0cb4ba8895cd" containerName="main"
Sep 08 15:09:16.197768 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.197701    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11a4810e-6ab5-4991-b426-38dde4220c68" containerName="main"
Sep 08 15:09:16.197768 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.197708    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="11a4810e-6ab5-4991-b426-38dde4220c68" containerName="main"
Sep 08 15:09:16.197865 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.197781    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4486d43b-4900-4152-8706-0cb4ba8895cd" containerName="main"
Sep 08 15:09:16.197865 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.197795    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="11a4810e-6ab5-4991-b426-38dde4220c68" containerName="main"
Sep 08 15:09:16.200686 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.200671    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.203422 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.203398    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gjjks\"/\"openshift-service-ca.crt\""
Sep 08 15:09:16.203545 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.203500    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gjjks\"/\"kube-root-ca.crt\""
Sep 08 15:09:16.203590 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.203578    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gjjks\"/\"default-dockercfg-vpdf8\""
Sep 08 15:09:16.212993 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.212970    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"]
Sep 08 15:09:16.214018 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.213991    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gm2hx_150e4cb2-f873-48a4-a41b-3dffe587ec1b/dns/0.log"
Sep 08 15:09:16.254276 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.254211    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gm2hx_150e4cb2-f873-48a4-a41b-3dffe587ec1b/kube-rbac-proxy/0.log"
Sep 08 15:09:16.361424 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.361400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-lib-modules\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.361583 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.361451    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-sys\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.361583 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.361509    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-podres\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.361583 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.361532    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-proc\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.361701 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.361584    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pj2l\" (UniqueName: \"kubernetes.io/projected/997535f6-6082-4919-93f7-1505379e2efe-kube-api-access-6pj2l\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.462091 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.462008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-lib-modules\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.462091 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.462059    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-sys\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.462302 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.462099    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-podres\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.462302 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.462121    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-proc\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.462302 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.462149    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pj2l\" (UniqueName: \"kubernetes.io/projected/997535f6-6082-4919-93f7-1505379e2efe-kube-api-access-6pj2l\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.462302 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.462182    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-lib-modules\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.462302 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.462216    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-proc\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.462302 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.462217    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-sys\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.462302 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.462246    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/997535f6-6082-4919-93f7-1505379e2efe-podres\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.466635 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.466618    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vsxtr_d88ea008-9ce7-4a83-abd8-bbde485345a3/dns-node-resolver/0.log"
Sep 08 15:09:16.470678 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.470660    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pj2l\" (UniqueName: \"kubernetes.io/projected/997535f6-6082-4919-93f7-1505379e2efe-kube-api-access-6pj2l\") pod \"perf-node-gather-daemonset-khd59\" (UID: \"997535f6-6082-4919-93f7-1505379e2efe\") " pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.510349 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.510329    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:16.781189 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:16.781162    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"]
Sep 08 15:09:16.783254 ip-10-0-137-6 kubenswrapper[2583]: W0908 15:09:16.783229    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod997535f6_6082_4919_93f7_1505379e2efe.slice/crio-2edd342ae6acf64eb075c864d377cf61fe982a9ce3207f07a8c4143b9fd38b83 WatchSource:0}: Error finding container 2edd342ae6acf64eb075c864d377cf61fe982a9ce3207f07a8c4143b9fd38b83: Status 404 returned error can't find the container with id 2edd342ae6acf64eb075c864d377cf61fe982a9ce3207f07a8c4143b9fd38b83
Sep 08 15:09:17.090149 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:17.090076    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hvzkl_2a15e5b5-3ba1-4587-943a-a4989e67b57b/node-ca/0.log"
Sep 08 15:09:17.456676 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:17.456580    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59" event={"ID":"997535f6-6082-4919-93f7-1505379e2efe","Type":"ContainerStarted","Data":"abc143251fa0a63f4d138eb1ee4ea0454fc9713eb4bd7f487a883508e2742ec8"}
Sep 08 15:09:17.456676 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:17.456642    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59" event={"ID":"997535f6-6082-4919-93f7-1505379e2efe","Type":"ContainerStarted","Data":"2edd342ae6acf64eb075c864d377cf61fe982a9ce3207f07a8c4143b9fd38b83"}
Sep 08 15:09:17.457099 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:17.456682    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:17.474455 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:17.474405    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59" podStartSLOduration=1.474388534 podStartE2EDuration="1.474388534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:17.472722705 +0000 UTC m=+2108.200539718" watchObservedRunningTime="2026-09-08 15:09:17.474388534 +0000 UTC m=+2108.202205591"
Sep 08 15:09:18.546278 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:18.546246    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dd8bb6964-pskvx_91ead5be-c3b9-4f61-931f-75199eeeae6f/kube-auth-proxy/0.log"
Sep 08 15:09:18.665461 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:18.665429    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-558h6_0ed5538c-88ce-4137-860a-1162e325b412/payload-pre-processing/0.log"
Sep 08 15:09:18.735625 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:18.735599    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65fd4f6866-nr2wr_fea5d1a3-561b-40e9-9978-9f1e662ef0c9/router/0.log"
Sep 08 15:09:19.575655 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:19.575627    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jj45j_6d9310f1-f184-4d94-a56b-bbceccbabe88/serve-healthcheck-canary/0.log"
Sep 08 15:09:20.257217 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:20.257186    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nzcsk_a3c361d8-f0d5-45fb-b228-11c76fbec590/kube-rbac-proxy/0.log"
Sep 08 15:09:20.281241 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:20.281211    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nzcsk_a3c361d8-f0d5-45fb-b228-11c76fbec590/exporter/0.log"
Sep 08 15:09:20.308466 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:20.308441    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nzcsk_a3c361d8-f0d5-45fb-b228-11c76fbec590/extractor/0.log"
Sep 08 15:09:22.537636 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:22.537598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjchd_b8d17862-2d5a-492b-a3e8-eb2b8c5eb2c6/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:22.968292 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:22.968216    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9pf4x_c91d268d-4b1a-4d68-ad3e-fc89ba9f4318/server/0.log"
Sep 08 15:09:23.339414 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:23.339381    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-766f77f8db-fdkgr_ae5ee9f8-55b4-4916-8cc4-a705216385e5/manager/0.log"
Sep 08 15:09:23.363938 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:23.363911    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-766f77f8db-fdkgr_ae5ee9f8-55b4-4916-8cc4-a705216385e5/copy-manifests/0.log"
Sep 08 15:09:23.473650 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:23.473621    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gjjks/perf-node-gather-daemonset-khd59"
Sep 08 15:09:24.449621 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:24.449580    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84fb6884f5-xddmw_eba0b560-b8e8-4b80-bddf-dcc9896dcae8/manager/0.log"
Sep 08 15:09:24.534662 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:24.534635    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dd8bb6964-pskvx_91ead5be-c3b9-4f61-931f-75199eeeae6f/kube-auth-proxy/0.log"
Sep 08 15:09:24.625583 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:24.625556    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-558h6_0ed5538c-88ce-4137-860a-1162e325b412/payload-pre-processing/0.log"
Sep 08 15:09:24.678385 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:24.678357    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65fd4f6866-nr2wr_fea5d1a3-561b-40e9-9978-9f1e662ef0c9/router/0.log"
Sep 08 15:09:26.854000 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:26.853967    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dd8bb6964-pskvx_91ead5be-c3b9-4f61-931f-75199eeeae6f/kube-auth-proxy/0.log"
Sep 08 15:09:26.940201 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:26.940174    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-558h6_0ed5538c-88ce-4137-860a-1162e325b412/payload-pre-processing/0.log"
Sep 08 15:09:26.990484 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:26.990460    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65fd4f6866-nr2wr_fea5d1a3-561b-40e9-9978-9f1e662ef0c9/router/0.log"
Sep 08 15:09:31.075665 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:31.075632    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wbz8_78adf8d2-113a-4251-a33f-66236876be03/kube-multus-additional-cni-plugins/0.log"
Sep 08 15:09:31.098090 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:31.098060    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wbz8_78adf8d2-113a-4251-a33f-66236876be03/egress-router-binary-copy/0.log"
Sep 08 15:09:31.127614 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:31.127590    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wbz8_78adf8d2-113a-4251-a33f-66236876be03/cni-plugins/0.log"
Sep 08 15:09:31.152361 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:31.152339    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wbz8_78adf8d2-113a-4251-a33f-66236876be03/bond-cni-plugin/0.log"
Sep 08 15:09:31.178717 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:31.178700    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wbz8_78adf8d2-113a-4251-a33f-66236876be03/routeoverride-cni/0.log"
Sep 08 15:09:31.200651 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:31.200628    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wbz8_78adf8d2-113a-4251-a33f-66236876be03/whereabouts-cni-bincopy/0.log"
Sep 08 15:09:31.220810 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:31.220789    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wbz8_78adf8d2-113a-4251-a33f-66236876be03/whereabouts-cni/0.log"
Sep 08 15:09:31.578798 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:31.578758    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dn9wg_328b6c1c-4a50-456f-9da9-795eba13313f/kube-multus/0.log"
Sep 08 15:09:31.749791 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:31.749749    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tmqvh_db5c60b0-c15e-496d-b2a0-8c76704ee8ae/network-metrics-daemon/0.log"
Sep 08 15:09:31.766182 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:31.766152    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tmqvh_db5c60b0-c15e-496d-b2a0-8c76704ee8ae/kube-rbac-proxy/0.log"
Sep 08 15:09:32.643040 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:32.643011    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9lt7h_55f744b4-526c-4826-8bc5-46ace246b9d7/ovn-controller/0.log"
Sep 08 15:09:32.660267 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:32.660243    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9lt7h_55f744b4-526c-4826-8bc5-46ace246b9d7/ovn-acl-logging/0.log"
Sep 08 15:09:32.686188 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:32.686164    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9lt7h_55f744b4-526c-4826-8bc5-46ace246b9d7/kube-rbac-proxy-node/0.log"
Sep 08 15:09:32.710944 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:32.710921    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9lt7h_55f744b4-526c-4826-8bc5-46ace246b9d7/kube-rbac-proxy-ovn-metrics/0.log"
Sep 08 15:09:32.733996 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:32.733976    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9lt7h_55f744b4-526c-4826-8bc5-46ace246b9d7/northd/0.log"
Sep 08 15:09:32.756717 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:32.756696    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9lt7h_55f744b4-526c-4826-8bc5-46ace246b9d7/nbdb/0.log"
Sep 08 15:09:32.779263 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:32.779248    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9lt7h_55f744b4-526c-4826-8bc5-46ace246b9d7/sbdb/0.log"
Sep 08 15:09:32.835728 ip-10-0-137-6 kubenswrapper[2583]: I0908 15:09:32.835705    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9lt7h_55f744b4-526c-4826-8bc5-46ace246b9d7/ovnkube-controller/0.log"